The class VideoEncoderSoftwareFallbackWrapper is an implementation
detail of webrtc/media/engine/webrtcvideoengine2.cc and should not be
directly under webrtc/video_encoder.h. The main purpose is to improve
the dependency graph in WebRTC so that VideoEncoderSoftwareFallbackWrapper
can depend on cricket::VideoCodec.
The test for VideoEncoderSoftwareFallbackWrapper is also moved from
webrtc/video/video_encoder_unittest.cc to
webrtc/media/engine/videoencodersoftwarefallbackwrapper_unittest.cc.
BUG=webrtc:6337
Review-Url: https://codereview.webrtc.org/2484863009
Cr-Commit-Position: refs/heads/master@{#15085}
Payload type -> RtpCodecParameters maps added for sender and receiver
side. It contains information that will be needed for RTCCodecStats[1]
dictionaries.
Video[Sender/Receiver]Info is updated with current codec payload type
for every stream which can be used to look up the codec in
VideoMediaInfo.
A similar change should be made for VoiceMediaInfo and
Voice[Sender/Receiver]Info.
[1] https://w3c.github.io/webrtc-stats/#codec-dict*
BUG=chromium:659117
Review-Url: https://codereview.webrtc.org/2484193002
Cr-Commit-Position: refs/heads/master@{#15060}
This CL will start to distinguish H264 profiles during SDP negotiation.
We currently don't look at the H264 profile at all and assume they are
all Constrained Baseline Level 3.1. This CL will start to check profiles
for equality when matching, and will generate the correct answer H264
level.
Each local supported H264 profile needs to be listed explicitly in the
list of local supported codecs, even if they are redundant. For example,
Baseline profile should be listed explicitly even though another profile
that is a superset of Baseline is also listed. The reason for this is to
simplify the code and avoid profile intersection during matching. So
VideoCodec::Matches will check for profile equality, and not check if
one codec is a subset of the other. This also leads to the nice property
that VideoCodec::Matches is symmetric, i.e. iif a.Matches(b) then
b.Matches(a).
BUG=webrtc:6337
TBR=tkchin@webrtc.org
Review-Url: https://codereview.webrtc.org/2483173002
Cr-Commit-Position: refs/heads/master@{#15051}
In older Chrome versions, the associated payload type in the RTX header
of retransmitted packets was always set to be the original media payload type,
regardless of the actual payload type of the packet. This meant that packets
encapsulated with RED headers had incorrect payload type information in the
RTX header. Due to an assumption in the receiver, this incorrect payload type
information would effectively be undone, leading to a working system.
Albeit working, this behaviour was undesired, and thus removed. In the interim,
several workarounds were introduced to not destroy interop between old and
new Chrome versions:
(1) https://codereview.webrtc.org/1649493004
- If no payload type mapping existed for RED over RTX, the payload type
of the underlying media would be used.
- If RED had been negotiated, received RTX packets would always be
assumed to contain RED.
(2) https://codereview.webrtc.org/1964473002
- If RED was removed from the remote description answer, it would be
disabled in the local receiver as well.
(3) https://codereview.webrtc.org/2033763002
- If RED was negotiated in the SDP, it would always be used, regardless
if ULPFEC was negotiated and used, or not.
Since the Chrome versions that exhibited the original bug now are very old,
this CL removes the workarounds from (1) and (2). In particular, after this
change, we will have the following behaviour:
- We assume that a payload type mapping for RED over RTX always is set.
If this is not the case, the RTX packet is not sent.
- The associated payload type of received RTX packets will always be obeyed.
- The (non)-existence of RED in the remote description does not affect the
local receiver.
The workaround in (3) still needs to exist, in order to interop with receivers
that did not have the workarounds in (1) and (2) removed. The change in (3)
can be removed in a couple of Chrome versions.
TESTED=Using AppRTC between patched Chrome (connected to ethernet) and standard Chrome M54 (connected to lossy internal Google WiFi), with and without FEC turned off using AppRTC flag. Also using "Munge SDP" sample on patched Chrome over loopback interface, with 100ms delay and 5% packet loss simulated using tc.
BUG=webrtc:6650
Review-Url: https://codereview.webrtc.org/2469093003
Cr-Commit-Position: refs/heads/master@{#15038}
Reason for revert:
This CL probably broke Chromium FYI.
Original issue's description:
> Stop caching supported codecs in WebRtcVideoEngine2
>
> We currently cache the result of GetSupportedCodecs in a member variable
> |video_codecs_| in WebRtcVideoEngine2. This means we need to keep
> |video_codecs_| and the result of GetSupportedCodecs in sync, which is
> error prone. It's simpler to just call GetSupportedCodecs when we need
> it, and we actually end up making fewer calls, so it's faster as well.
> This CL also returns all std::vectors by-value instead of by-ref. Move
> semantic together with in-place filtering of codecs actually end up with
> fewer copies, and it's also simpler to not return references.
>
> BUG=webrtc:6337
>
> Committed: https://crrev.com/9f71ec5a3e3175751f4475b126cfda89767363f2
> Cr-Commit-Position: refs/heads/master@{#15007}
TBR=tommi@webrtc.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG=webrtc:6337
Review-Url: https://codereview.webrtc.org/2489173004
Cr-Commit-Position: refs/heads/master@{#15014}
We currently cache the result of GetSupportedCodecs in a member variable
|video_codecs_| in WebRtcVideoEngine2. This means we need to keep
|video_codecs_| and the result of GetSupportedCodecs in sync, which is
error prone. It's simpler to just call GetSupportedCodecs when we need
it, and we actually end up making fewer calls, so it's faster as well.
This CL also returns all std::vectors by-value instead of by-ref. Move
semantic together with in-place filtering of codecs actually end up with
fewer copies, and it's also simpler to not return references.
BUG=webrtc:6337
Review-Url: https://codereview.webrtc.org/2492473002
Cr-Commit-Position: refs/heads/master@{#15007}
This CL removes WebRtcVideoReceiveStream::FilterSupportedCodecs and
adds more fine grained code for selecting and verifying codecs. This
also removes unnecessary copying of cricket::VideoCodecs.
BUG=webrtc:6337
Review-Url: https://codereview.webrtc.org/2474433012
Cr-Commit-Position: refs/heads/master@{#14964}
Suppress WebRtcVideoEncoderFactory overloaded virtual function warning
in WebRtcSimulcastEncoderFactory and FakeWebRtcVideoEncoderFactory.
This warning is triggered by the change in this CL:
https://codereview.webrtc.org/2449993003/.
BUG=webrtc:6402, webrtc:6337
Review-Url: https://codereview.webrtc.org/2468253002
Cr-Commit-Position: refs/heads/master@{#14901}
This removes the VideoSendStream::LoadObserver interface and the implementation in WebrtcVideoSendStream and replace it with VideoSinkWants through the VideoSourceInterface.
To do that that, some stats for CPU adaptation is moved into VideoSendStream. Also handling of the CVO rtp header extension is moved to VideoSendStreamImpl.
BUG=webrtc:5687
TBR=mflodman@webrtc.org
Review-Url: https://codereview.webrtc.org/2304363002
Cr-Commit-Position: refs/heads/master@{#14877}
This CL introduces two new functions to the WebRtcVideoEncoderFactory
interface based on cricket::VideoFormat instead of
WebRtcVideoEncoderFactory::VideoCodec. The functions are:
WebRtcVideoEncoderFactory::CreateVideoEncoder() and
WebRtcVideoEncoderFactory::supported_codecs(). In order to make a smooth
transition to the new interface, the old functions are kept, and default
implementations are provided for both the old and new functions so that
external clients can switch from the old to the new functions in peace.
The default implementations will just convert between
cricket::VideoFormat and WebRtcVideoEncoderFactory::VideoCodec. Once all
external clients have updated their code, the plan is to remove the old
functions and all default implementations to make
WebRtcVideoEncoderFactory a pure interface again.
BUG=webrtc:6402,webrtc:6337
Review-Url: https://codereview.webrtc.org/2449993003
Cr-Commit-Position: refs/heads/master@{#14826}
Since WebRtcVideoSendStream have reconfigures the send codec to match the incoming captured frames widht and height they have not been used.
Framerate has just been set when parsing sdp to 60fps and not changed elsewhere.
This cl require some upstream projects to change first.
BUG=webrtc:5332
Review-Url: https://codereview.webrtc.org/2408153002
Cr-Commit-Position: refs/heads/master@{#14733}
all methods but a few constructors. And similarly for the
subclass cricket::WebRtcVideoFrame.
TBR=tkchin@webrtc.org # Added an include line
BUG=webrtc:5682
Review-Url: https://codereview.webrtc.org/2315663002
Cr-Commit-Position: refs/heads/master@{#14576}
Also rename some related minor methods. No functional changes
are intended/expected.
BUG=webrtc:5654
Review-Url: https://codereview.webrtc.org/2391963002
Cr-Commit-Position: refs/heads/master@{#14513}
The original landed cl is in patchset 1.
The following patchset fix VideoQualityTest as well as fix the case where max_bitrate is set in the SendParams. A unit test is added for that as well.
Original cl description:
Let ViEEncoder handle resolution changes.
This cl move codec reconfiguration due to video frame size changes from WebRtcVideoSendStream to ViEEncoder.
With this change, many variables in WebRtcVideoSendStream no longer need to be locked.
BUG=webrtc:5687, webrtc:6371, webrtc:5332
Review-Url: https://codereview.webrtc.org/2386573002
Cr-Commit-Position: refs/heads/master@{#14467}
This cl move codec reconfiguration due to video frame size changes from WebRtcVideoSendStream to ViEEncoder.
With this change, many variables in WebRtcVideoSendStream no longer need to be locked.
BUG=webrtc:5687, webrtc:6371, webrtc:5332
Review-Url: https://codereview.webrtc.org/2351633002
Cr-Commit-Position: refs/heads/master@{#14445}
This cl move calculation of stats for prefered_media_bitrate_bps from webrtcvideoengine2.GetStats to SendStatisticsProxy::OnEncoderReconfigured.
This aligns better with how other send stats are reported and is needed as a prerequisite for moving video encoder configuration due to video resolution change
from WebRtcVideoEngine2 to ViEEncoder.
BUG=webrtc:6371
R=mflodman@webrtc.org, sprang@webrtc.org
Review URL: https://codereview.webrtc.org/2368223002 .
Cr-Commit-Position: refs/heads/master@{#14431}
Original description:
Add proper lifetime of encoder-specific settings.
Permits passing VideoEncoderConfig between threads and not worry about
the lifetime of an underlying void pointer. Also adds type safety to
unpacking of codec-specific settings.
These settings are not yet propagating to VideoEncoder interfaces, but
the aim is to get rid of webrtc::VideoCodec for VideoEncoder.
BUG=webrtc:3424
R=perkj@webrtc.org, pbos@webrtc.orgTBR=mflodman@webrtc.org
Review-Url: https://codereview.webrtc.org/2347843002
Cr-Commit-Position: refs/heads/master@{#14396}
Adds new method VideoSendStream::SetSource(rtc::VideoSourceInterface* and VieEncoder::SetSource(rtc::VideoSourceInterface*)
This is the first step needed in order for the ViEEncoder to request downscaling using rtc::VideoSinkWants instead of separately reporting CPU overuse and internally doing downscaling due to QP values
This cl
Revert "Revert of Replace interface VideoCapturerInput with VideoSinkInterface. (patchset #13 id:280001 of https://codereview.webrtc.org/2257413002/ )"
This reverts commit 9fdbda6aa3f66ea872344c22e79b23361047cbab.
and fix the problem in the original cl in video_quality_test.cc
BUG=webrtc:5687
TBR=mflodman@webrtc.org
Review-Url: https://codereview.webrtc.org/2348533002
Cr-Commit-Position: refs/heads/master@{#14265}
Reason for revert:
Fails on Mac and Linux webrtc_perf_tests
Original issue's description:
> Replace VideoCapturerInput with VideoSinkInterface.
> Adds new method VideoSendStream::SetSource(rtc::VideoSourceInterface* and VieEncoder::SetSource(rtc::VideoSourceInterface*)
>
> This is the first step needed in order for the ViEEncoder to request downscaling using rtc::VideoSinkWants instead of separately reporting CPU overuse and internally doing downscaling due to QP values.
>
> BUG=webrtc:5687
> // Android CQ seems broken.
> NOTRY=true
>
> Committed: https://crrev.com/95a226f55ae7e32b83a6ba96232fb105a014dc6c
> Cr-Commit-Position: refs/heads/master@{#14238}
TBR=nisse@webrtc.org,sprang@webrtc.org,mflodman@webrtc.org,stefan@webrtc.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG=webrtc:5687
Review-Url: https://codereview.webrtc.org/2344923002
Cr-Commit-Position: refs/heads/master@{#14239}
Adds new method VideoSendStream::SetSource(rtc::VideoSourceInterface* and VieEncoder::SetSource(rtc::VideoSourceInterface*)
This is the first step needed in order for the ViEEncoder to request downscaling using rtc::VideoSinkWants instead of separately reporting CPU overuse and internally doing downscaling due to QP values.
BUG=webrtc:5687
// Android CQ seems broken.
NOTRY=true
Review-Url: https://codereview.webrtc.org/2257413002
Cr-Commit-Position: refs/heads/master@{#14238}
RecreateWebRtcStream was checking the "sending_" flag, but wasn't
checking rtp_parameters_.encodings[0].active. As a result, if an
application calls "RtpSender.setParameters(inactive_params)" then later
the stream is recreated due to some change in SDP parameters, the stream
would incorrectly start sending.
R=pthatcher@webrtc.org, skvlad@webrtc.org
Review URL: https://codereview.webrtc.org/2246313002 .
Cr-Commit-Position: refs/heads/master@{#14116}
Replaces render_time_ms_, but old accessors are kept for
compatibility.
Also short-circuit timestamp translation in
WebRtcVideoChannel2::WebRtcVideoSendStream::OnFrame.
BUG=webrtc:5682, webrtc:5740
Review-Url: https://codereview.webrtc.org/2282713002
Cr-Commit-Position: refs/heads/master@{#14062}
cl was originally reviewed here:
https://codereview.webrtc.org/2060403002/
- Add task queue to Call with the intent of replacing the use of one of the process threads.
- Split VideoSendStream in two. VideoSendStreamInternal is created and used on the new task queue.
- BitrateAllocator is now created on libjingle's worker thread but always used on the new task queue instead of both encoder threads and the process thread.
- VideoEncoderConfig and VideoSendStream::Config support move semantics.
- The encoder thread is moved from VideoSendStream to ViEEncoder. Frames are forwarded directly to ViEEncoder which is responsible for timestamping ? and encoding the frames.
TBR=mflodman@webrtc.org
BUG=webrtc:5687
Review-Url: https://codereview.webrtc.org/2250123002
Cr-Commit-Position: refs/heads/master@{#14014}
Reason for revert:
Failed on Win 10 Chrome FYI.
https://build.chromium.org/p/chromium.webrtc.fyi/builders/Win10%20Tester/builds/3847/steps/content_browsertests/logs/stdio
#
# Fatal error in e:\b\c\b\win_builder\src\third_party\webrtc\base\task_queue_win.cc, line 138
# last system error: 87
# Check failed: ((DWORD)0xFFFFFFFF) != result (4294967295 vs. 4294967295)
#
WebRtcBrowserTest
#
Original issue's description:
> - Add task queue to Call with the intent of replacing the use of one of the process threads.
>
> - Split VideoSendStream in two. VideoSendStreamInternal is created and used on the new task queue.
>
> - BitrateAllocator is now created on libjingle's worker thread but always used on the new task queue instead of both encoder threads and the process thread.
>
> - VideoEncoderConfig and VideoSendStream::Config support move semantics.
>
> - The encoder thread is moved from VideoSendStream to ViEEncoder. Frames are forwarded directly to ViEEncoder which is responsible for timestamping ? and encoding the frames.
>
> BUG=webrtc:5687
>
> Committed: https://crrev.com/cc168360f41322332860cb075edeb1cde21aa473
> Cr-Commit-Position: refs/heads/master@{#13767}
TBR=tommi@webrtc.org,mflodman@webrtc.org,stefan@webrtc.org,sprang@webrtc.org,pbos@webrtc.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG=webrtc:5687
Review-Url: https://codereview.webrtc.org/2248713003
Cr-Commit-Position: refs/heads/master@{#13774}
- Split VideoSendStream in two. VideoSendStreamInternal is created and used on the new task queue.
- BitrateAllocator is now created on libjingle's worker thread but always used on the new task queue instead of both encoder threads and the process thread.
- VideoEncoderConfig and VideoSendStream::Config support move semantics.
- The encoder thread is moved from VideoSendStream to ViEEncoder. Frames are forwarded directly to ViEEncoder which is responsible for timestamping ? and encoding the frames.
BUG=webrtc:5687
Review-Url: https://codereview.webrtc.org/2060403002
Cr-Commit-Position: refs/heads/master@{#13767}
Passing transport_frame_id() to VideoSink will allow to identify incoming video
frames, which will make it possible to correlate video frames on the
sender and on the receiver.
BUG=chromium:621691
R=mflodman@webrtc.org, stefan@webrtc.org
Review URL: https://codereview.webrtc.org/2088953002 .
Cr-Commit-Position: refs/heads/master@{#13596}
VideoDecoderParams contains the id of the receive video
stream. Motivation behind this change is to enable down
stream apps easier map raw non-decoded data to incoming
streams.
BUG=b/28636393
Review-Url: https://codereview.webrtc.org/2052233002
Cr-Commit-Position: refs/heads/master@{#13250}
Don't use VideoFrameBuffer::MutableDataY and friends, instead, use
I420Buffer::SetToBlack.
Also introduce static method I420Buffer::Create, to create an object and
return a scoped_refptr.
TBR=marpan@webrtc.org # Trivial change to video_denoiser.cc
BUG=webrtc:5921
Review-Url: https://codereview.webrtc.org/2078943002
Cr-Commit-Position: refs/heads/master@{#13212}
This change reduces the number of times the Android hardware video
encoder is reconfigured when making an outgoing call. With this change,
the encoder should only be initialized once as opposed to the ~3 times
it happens currently.
Before the fix, the following sequence of events caused the extra
reconfigurations:
1. After the SetLocalDescription call, the WebRtcVideoSendStream is created.
All frames from the camera are dropped until the corresponding
VideoSendStream is created.
2. SetRemoteDescription() triggers the VideoSendStream creation. At
this point, the encoder is configured for the first time, with the
frame dimensions set to a low resolution default (176x144).
3. When the first video frame is received from the camera after the
VideoSendStreamIsCreated, the encoder is reconfigured to the correct
dimensions. If we are using the Android hardware encoder, the default
configuration is set to encode from a memory buffer (use_surface=false).
4. When the frame is passed down to the encoder in
androidmediaencoder_jni.cc EncodeOnCodecThread(), it may be stored in
a texture instead of a memory buffer. In this case, yet another
reconfiguration takes place to enable encoding from a texture.
5. Even if the resolution and texture flag were known at the start of
the call, there would be a reconfiguration involved if the camera is
rotated (such as when making a call from a phone in portrait orientation).
The reason for that is that at construction time, WebRtcVideoEngine2
sets the VideoSinkWants structure parameter to request frames rotated
by the source; the early frames will then arrive in portrait resolution.
When the remote description is finally set, if the rotation RTP extension
is supported by the remote receiver, the source is asked to provide
non-rotated frames. The very next frame will then arrive in landscape
resolution with a non-zero rotation value to be applied by the receiver.
Since the encoder was configured with the last (portrait) frame size,
it's going to need to be reconfigured again.
The fix makes the following changes:
1. WebRtcVideoSendStream::OnFrame() now caches the last seen frame
dimensions, and whether the frame was stored in a texture.
2. When the encoder is configured the first time
(WebRtcVideoSendStream::SetCodec()) - the last seen frame dimensions
are used instead of the default dimensions.
3. A flag that indicates if encoding is to be done from a texture has
been added to the webrtc::VideoStream and webrtc::VideoCodec structs,
and it's been wired up to be passed down all the way to the JNI code in
androidmediaencoder_jni.cc.
4. MediaCodecVideoEncoder::InitEncode is now reading the is_surface
flag from the VideoCodec structure instead of guessing the default as
false. This way we end up with the correct encoder configuration the
first time around.
5. WebRtcVideoSendStream now takes an optimistic guess and requests non-
rotated frames when the supported RtpExtensions list is not available.
This makes the "early" frames arrive non-rotated, and the cached dimensions
will be correct for the common case when the rotation extension is supported.
If the other side is an older endpoint which does not support rotation,
the encoder will have to be reconfigured - but it's better to penalize the
uncommon case rather than the common one.
Review-Url: https://codereview.webrtc.org/2067103002
Cr-Commit-Position: refs/heads/master@{#13173}
Every message will now be traced with the location from which it was
posted, including function name, file and line number.
This CL also writes a normal LOG message when the dispatch took more
than a certain amount of time (currently 50ms).
This logging should help us identify messages that are taking
longer than expected to be dispatched.
R=pthatcher@webrtc.org, tommi@webrtc.org
Review URL: https://codereview.webrtc.org/2019423006 .
Cr-Commit-Position: refs/heads/master@{#13104}
