FlexfecSender is owned and configured by VideoSendStream.
BUG=webrtc:5654
Review-Url: https://codereview.webrtc.org/2501503003
Cr-Commit-Position: refs/heads/master@{#15082}
In older Chrome versions, the associated payload type in the RTX header
of retransmitted packets was always set to be the original media payload type,
regardless of the actual payload type of the packet. This meant that packets
encapsulated with RED headers had incorrect payload type information in the
RTX header. Due to an assumption in the receiver, this incorrect payload type
information would effectively be undone, leading to a working system.
Albeit working, this behaviour was undesired, and thus removed. In the interim,
several workarounds were introduced to not destroy interop between old and
new Chrome versions:
(1) https://codereview.webrtc.org/1649493004
- If no payload type mapping existed for RED over RTX, the payload type
of the underlying media would be used.
- If RED had been negotiated, received RTX packets would always be
assumed to contain RED.
(2) https://codereview.webrtc.org/1964473002
- If RED was removed from the remote description answer, it would be
disabled in the local receiver as well.
(3) https://codereview.webrtc.org/2033763002
- If RED was negotiated in the SDP, it would always be used, regardless
if ULPFEC was negotiated and used, or not.
Since the Chrome versions that exhibited the original bug now are very old,
this CL removes the workarounds from (1) and (2). In particular, after this
change, we will have the following behaviour:
- We assume that a payload type mapping for RED over RTX always is set.
If this is not the case, the RTX packet is not sent.
- The associated payload type of received RTX packets will always be obeyed.
- The (non)-existence of RED in the remote description does not affect the
local receiver.
The workaround in (3) still needs to exist, in order to interop with receivers
that did not have the workarounds in (1) and (2) removed. The change in (3)
can be removed in a couple of Chrome versions.
TESTED=Using AppRTC between patched Chrome (connected to ethernet) and standard Chrome M54 (connected to lossy internal Google WiFi), with and without FEC turned off using AppRTC flag. Also using "Munge SDP" sample on patched Chrome over loopback interface, with 100ms delay and 5% packet loss simulated using tc.
BUG=webrtc:6650
Review-Url: https://codereview.webrtc.org/2469093003
Cr-Commit-Position: refs/heads/master@{#15038}
- Change const ptr to const ref in parameter list.
Using nullptr as argument was invalid, so no need to send
pointer instead of reference.
- Change return type to void or bool, where appropriate
BUG=webrtc:5654
Review-Url: https://codereview.webrtc.org/2455963003
Cr-Commit-Position: refs/heads/master@{#14945}
Prior to this change, we signalled that ULPFEC was disabled
through a bool, but that RED was disabled by setting its
payload type to -1. The latter is consistent with how we
disable RED/ULPFEC in the config, so this CL removes the
ULPFEC bool from the {,Set}UlpfecConfig chain of member
functions.
BUG=webrtc:5654
Review-Url: https://codereview.webrtc.org/2460533002
Cr-Commit-Position: refs/heads/master@{#14944}
At the same time, change to using int's instead of uint8_t's for the payload type.
This allows us to signal disabled FEC or RED using the sentinel value -1, which
is commonplace in other parts of the code.
These APIs will be deprecated when ULPFEC is deprecated.
BUG=webrtc:5654
Review-Url: https://codereview.webrtc.org/2448463003
Cr-Commit-Position: refs/heads/master@{#14942}
This removes the VideoSendStream::LoadObserver interface and the implementation in WebrtcVideoSendStream and replace it with VideoSinkWants through the VideoSourceInterface.
To do that that, some stats for CPU adaptation is moved into VideoSendStream. Also handling of the CVO rtp header extension is moved to VideoSendStreamImpl.
BUG=webrtc:5687
TBR=mflodman@webrtc.org
Review-Url: https://codereview.webrtc.org/2304363002
Cr-Commit-Position: refs/heads/master@{#14877}
The renaming is to reflect this class is only used for RTCP interaction
and not for other transports.
This Cl will be followed by multiple CLs moving all send-side RTP
functionality to a separate class, rtp module ownership away from
VideoSendStream and use TaskQueue instead of ProcessThread for RTP.
BUG=webrtc:6456
Review-Url: https://codereview.webrtc.org/2390463002
Cr-Commit-Position: refs/heads/master@{#14556}
Also rename some related minor methods. No functional changes
are intended/expected.
BUG=webrtc:5654
Review-Url: https://codereview.webrtc.org/2391963002
Cr-Commit-Position: refs/heads/master@{#14513}
The implementation is borrowed from Chromium.
Also change use of raw pointers in VideoSendStreamImpl to use WeakPtr<>
BUG= webrtc:6289
Review-Url: https://codereview.webrtc.org/2367853002
Cr-Commit-Position: refs/heads/master@{#14468}
The original landed cl is in patchset 1.
The following patchset fix VideoQualityTest as well as fix the case where max_bitrate is set in the SendParams. A unit test is added for that as well.
Original cl description:
Let ViEEncoder handle resolution changes.
This cl move codec reconfiguration due to video frame size changes from WebRtcVideoSendStream to ViEEncoder.
With this change, many variables in WebRtcVideoSendStream no longer need to be locked.
BUG=webrtc:5687, webrtc:6371, webrtc:5332
Review-Url: https://codereview.webrtc.org/2386573002
Cr-Commit-Position: refs/heads/master@{#14467}
This cl move codec reconfiguration due to video frame size changes from WebRtcVideoSendStream to ViEEncoder.
With this change, many variables in WebRtcVideoSendStream no longer need to be locked.
BUG=webrtc:5687, webrtc:6371, webrtc:5332
Review-Url: https://codereview.webrtc.org/2351633002
Cr-Commit-Position: refs/heads/master@{#14445}
This cl move calculation of stats for prefered_media_bitrate_bps from webrtcvideoengine2.GetStats to SendStatisticsProxy::OnEncoderReconfigured.
This aligns better with how other send stats are reported and is needed as a prerequisite for moving video encoder configuration due to video resolution change
from WebRtcVideoEngine2 to ViEEncoder.
BUG=webrtc:6371
R=mflodman@webrtc.org, sprang@webrtc.org
Review URL: https://codereview.webrtc.org/2368223002 .
Cr-Commit-Position: refs/heads/master@{#14431}
This cl change so that all encoder configuration changes are reported to VideoSendStream through the ViEEncoder.
Also, the PayLoadRouter is changed to never stop sending on a an ssrc due to the encoder video frame size changes. Instead, the number of sending streams is only decided by the number of sending ssrc.
This cl is a preparation for moving encoder reconfiguration due to input video frame size changes from WebRtcVideoSendStream to ViEEncoder.
BUG=webrtc:5687, webrtc:6371
R=mflodman@webrtc.org
Review URL: https://codereview.webrtc.org/2338133003 .
Cr-Commit-Position: refs/heads/master@{#14371}
Adds new method VideoSendStream::SetSource(rtc::VideoSourceInterface* and VieEncoder::SetSource(rtc::VideoSourceInterface*)
This is the first step needed in order for the ViEEncoder to request downscaling using rtc::VideoSinkWants instead of separately reporting CPU overuse and internally doing downscaling due to QP values
This cl
Revert "Revert of Replace interface VideoCapturerInput with VideoSinkInterface. (patchset #13 id:280001 of https://codereview.webrtc.org/2257413002/ )"
This reverts commit 9fdbda6aa3f66ea872344c22e79b23361047cbab.
and fix the problem in the original cl in video_quality_test.cc
BUG=webrtc:5687
TBR=mflodman@webrtc.org
Review-Url: https://codereview.webrtc.org/2348533002
Cr-Commit-Position: refs/heads/master@{#14265}
Reason for revert:
Fails on Mac and Linux webrtc_perf_tests
Original issue's description:
> Replace VideoCapturerInput with VideoSinkInterface.
> Adds new method VideoSendStream::SetSource(rtc::VideoSourceInterface* and VieEncoder::SetSource(rtc::VideoSourceInterface*)
>
> This is the first step needed in order for the ViEEncoder to request downscaling using rtc::VideoSinkWants instead of separately reporting CPU overuse and internally doing downscaling due to QP values.
>
> BUG=webrtc:5687
> // Android CQ seems broken.
> NOTRY=true
>
> Committed: https://crrev.com/95a226f55ae7e32b83a6ba96232fb105a014dc6c
> Cr-Commit-Position: refs/heads/master@{#14238}
TBR=nisse@webrtc.org,sprang@webrtc.org,mflodman@webrtc.org,stefan@webrtc.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG=webrtc:5687
Review-Url: https://codereview.webrtc.org/2344923002
Cr-Commit-Position: refs/heads/master@{#14239}
Adds new method VideoSendStream::SetSource(rtc::VideoSourceInterface* and VieEncoder::SetSource(rtc::VideoSourceInterface*)
This is the first step needed in order for the ViEEncoder to request downscaling using rtc::VideoSinkWants instead of separately reporting CPU overuse and internally doing downscaling due to QP values.
BUG=webrtc:5687
// Android CQ seems broken.
NOTRY=true
Review-Url: https://codereview.webrtc.org/2257413002
Cr-Commit-Position: refs/heads/master@{#14238}
cl was originally reviewed here:
https://codereview.webrtc.org/2060403002/
- Add task queue to Call with the intent of replacing the use of one of the process threads.
- Split VideoSendStream in two. VideoSendStreamInternal is created and used on the new task queue.
- BitrateAllocator is now created on libjingle's worker thread but always used on the new task queue instead of both encoder threads and the process thread.
- VideoEncoderConfig and VideoSendStream::Config support move semantics.
- The encoder thread is moved from VideoSendStream to ViEEncoder. Frames are forwarded directly to ViEEncoder which is responsible for timestamping ? and encoding the frames.
TBR=mflodman@webrtc.org
BUG=webrtc:5687
Review-Url: https://codereview.webrtc.org/2250123002
Cr-Commit-Position: refs/heads/master@{#14014}
Reason for revert:
Failed on Win 10 Chrome FYI.
https://build.chromium.org/p/chromium.webrtc.fyi/builders/Win10%20Tester/builds/3847/steps/content_browsertests/logs/stdio
#
# Fatal error in e:\b\c\b\win_builder\src\third_party\webrtc\base\task_queue_win.cc, line 138
# last system error: 87
# Check failed: ((DWORD)0xFFFFFFFF) != result (4294967295 vs. 4294967295)
#
WebRtcBrowserTest
#
Original issue's description:
> - Add task queue to Call with the intent of replacing the use of one of the process threads.
>
> - Split VideoSendStream in two. VideoSendStreamInternal is created and used on the new task queue.
>
> - BitrateAllocator is now created on libjingle's worker thread but always used on the new task queue instead of both encoder threads and the process thread.
>
> - VideoEncoderConfig and VideoSendStream::Config support move semantics.
>
> - The encoder thread is moved from VideoSendStream to ViEEncoder. Frames are forwarded directly to ViEEncoder which is responsible for timestamping ? and encoding the frames.
>
> BUG=webrtc:5687
>
> Committed: https://crrev.com/cc168360f41322332860cb075edeb1cde21aa473
> Cr-Commit-Position: refs/heads/master@{#13767}
TBR=tommi@webrtc.org,mflodman@webrtc.org,stefan@webrtc.org,sprang@webrtc.org,pbos@webrtc.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG=webrtc:5687
Review-Url: https://codereview.webrtc.org/2248713003
Cr-Commit-Position: refs/heads/master@{#13774}
- Split VideoSendStream in two. VideoSendStreamInternal is created and used on the new task queue.
- BitrateAllocator is now created on libjingle's worker thread but always used on the new task queue instead of both encoder threads and the process thread.
- VideoEncoderConfig and VideoSendStream::Config support move semantics.
- The encoder thread is moved from VideoSendStream to ViEEncoder. Frames are forwarded directly to ViEEncoder which is responsible for timestamping ? and encoding the frames.
BUG=webrtc:5687
Review-Url: https://codereview.webrtc.org/2060403002
Cr-Commit-Position: refs/heads/master@{#13767}
These arguments are not really known when calling SetEncodingData. They are still provided as argument to ProtectionBitrateCalculator::SetTargetRates though.
This cl is broken out from https://codereview.webrtc.org/2060403002/
BUG=webrtc:5687
Review-Url: https://codereview.webrtc.org/2121983002
Cr-Commit-Position: refs/heads/master@{#13429}
Reason for revert:
Breaks upstream code.
Original issue's description:
> Refactor NACK bitrate allocation
>
> Nack bitrate allocation should not be done on a per-rtp-module basis,
> but rather shared bitrate pool per call. This CL moves allocation to the
> pacer and cleans up a bunch if bitrate stats handling.
>
> BUG=
> R=danilchap@webrtc.org, stefan@webrtc.org, tommi@webrtc.org
>
> Committed: 5fc59e810bTBR=tommi@webrtc.org,danilchap@webrtc.org,stefan@webrtc.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG=
Review-Url: https://codereview.webrtc.org/2131913003
Cr-Commit-Position: refs/heads/master@{#13417}
A recent refactoring (r13192) introduced a bug where the min transmit
config wasn't being respected. Specifically, if a VideoSendStream was
created without it and the reconfigured, the min transmit bitrate would
not take effect. Probably the other way around as well.
BUG=webrtc::5687
Review-Url: https://codereview.webrtc.org/2106183002
Cr-Commit-Position: refs/heads/master@{#13390}
When the target bitrate is zero, currently VideoSendStream.Stats.target_media_bitrate_bps show the last set rate before the target was set to zero.
BUG=webrtc::5687 b/29574845
Review-Url: https://codereview.webrtc.org/2122743003
Cr-Commit-Position: refs/heads/master@{#13386}
This CL changes the auto-pause logic to suspend a stream based on the
encoder target bitrate instead of the allocated bitrate for a stream,
to account for possible protection, e.g. FEC and NACK.
This CL also adds periodic logging of the current BWE and possibility
to run with suspension in video loopback test.
BUG=webrtc:5868
R=stefan@webrtc.org
Review URL: https://codereview.webrtc.org/2117493002 .
Cr-Commit-Position: refs/heads/master@{#13360}
This cl change so that VideoSendStream::Start adds the stream as a BitrateObserver and VideoSendStream::Stop removes the stream as observer.
That also means that start will trigger a VideoEncoder::SetRate call with the most recent bitrate estimate.
VideoSendStream::Stop will trigger a VideoEncoder::SetRate with bitrate = 0.
BUG=webrtc:5687 b/28636240
Review-Url: https://codereview.webrtc.org/2070343002
Cr-Commit-Position: refs/heads/master@{#13192}
This change reduces the number of times the Android hardware video
encoder is reconfigured when making an outgoing call. With this change,
the encoder should only be initialized once as opposed to the ~3 times
it happens currently.
Before the fix, the following sequence of events caused the extra
reconfigurations:
1. After the SetLocalDescription call, the WebRtcVideoSendStream is created.
All frames from the camera are dropped until the corresponding
VideoSendStream is created.
2. SetRemoteDescription() triggers the VideoSendStream creation. At
this point, the encoder is configured for the first time, with the
frame dimensions set to a low resolution default (176x144).
3. When the first video frame is received from the camera after the
VideoSendStreamIsCreated, the encoder is reconfigured to the correct
dimensions. If we are using the Android hardware encoder, the default
configuration is set to encode from a memory buffer (use_surface=false).
4. When the frame is passed down to the encoder in
androidmediaencoder_jni.cc EncodeOnCodecThread(), it may be stored in
a texture instead of a memory buffer. In this case, yet another
reconfiguration takes place to enable encoding from a texture.
5. Even if the resolution and texture flag were known at the start of
the call, there would be a reconfiguration involved if the camera is
rotated (such as when making a call from a phone in portrait orientation).
The reason for that is that at construction time, WebRtcVideoEngine2
sets the VideoSinkWants structure parameter to request frames rotated
by the source; the early frames will then arrive in portrait resolution.
When the remote description is finally set, if the rotation RTP extension
is supported by the remote receiver, the source is asked to provide
non-rotated frames. The very next frame will then arrive in landscape
resolution with a non-zero rotation value to be applied by the receiver.
Since the encoder was configured with the last (portrait) frame size,
it's going to need to be reconfigured again.
The fix makes the following changes:
1. WebRtcVideoSendStream::OnFrame() now caches the last seen frame
dimensions, and whether the frame was stored in a texture.
2. When the encoder is configured the first time
(WebRtcVideoSendStream::SetCodec()) - the last seen frame dimensions
are used instead of the default dimensions.
3. A flag that indicates if encoding is to be done from a texture has
been added to the webrtc::VideoStream and webrtc::VideoCodec structs,
and it's been wired up to be passed down all the way to the JNI code in
androidmediaencoder_jni.cc.
4. MediaCodecVideoEncoder::InitEncode is now reading the is_surface
flag from the VideoCodec structure instead of guessing the default as
false. This way we end up with the correct encoder configuration the
first time around.
5. WebRtcVideoSendStream now takes an optimistic guess and requests non-
rotated frames when the supported RtpExtensions list is not available.
This makes the "early" frames arrive non-rotated, and the cached dimensions
will be correct for the common case when the rotation extension is supported.
If the other side is an older endpoint which does not support rotation,
the encoder will have to be reconfigured - but it's better to penalize the
uncommon case rather than the common one.
Review-Url: https://codereview.webrtc.org/2067103002
Cr-Commit-Position: refs/heads/master@{#13173}
1. It moves calculation of the needed padding to VideoSendStream instead of ViEEncoder and only does it once per send Stream instead of every time the network estimate changes.
2. The maximum amount of padding sent was prior to this cl calculated and updated based on network estimate changes. However, it can only change based on encoder configuration changes and if send streams are added or removed. This cl change the VideoSendStream/VieEncoder to notify the BitrateAllocator of changes to the needed padding bitrate and for BitrateAllocator to notify Call of these changes.
3. Fixed an issue in the SendPacer where it could send a padding packet before sending a real packet. This caused the test EndToEndTest.RestartingSendStreamPreservesRtpStatesWithRtx to fail with these refactorings since the pacer suddenly could send a padding packet before the encoder had produced its first frame.
BUG=webrtc:5687
Review-Url: https://codereview.webrtc.org/1993113003
Cr-Commit-Position: refs/heads/master@{#13149}
EncodedFrameCallbackAdapter was used VideoSendStream and
VideoReceiveStream, but there is no reason to have it as these classes
can call EncodedFrameObserver directly.
Review-Url: https://codereview.webrtc.org/2068463004
Cr-Commit-Position: refs/heads/master@{#13145}
This CL implements auto pausing video streams per stream with logic to
avoid toggling state too often.
Also re-enabling tests disabled for Mac, with the assumption the new
logic removes flakiness.
BUG=webrtc:5868,webrtc:5407
R=pbos@webrtc.org, stefan@webrtc.org
Review URL: https://codereview.webrtc.org/2035383002 .
Cr-Commit-Position: refs/heads/master@{#13092}
This shouldn't be needed, but because the receiver assumes RTX packets
contain RED if configured to receive them (due to an incompatibility
issue), we also have to make sure we send them for now.
BUG=webrtc:5675
Review-Url: https://codereview.webrtc.org/2033763002
Cr-Commit-Position: refs/heads/master@{#13024}
This cl split the class MediaOptimization into two parts. One that deals with frame dropping and stats and one new class called ProtectionBitrateCalculator that deals with calculating the needed FEC parameters and how much of the estimated network bitrate that can be used by an encoder
Note that the logic of how FEC and the needed bitrates is not changed.
BUG=webrtc:5687
R=asapersson@webrtc.org, stefan@webrtc.org
Review URL: https://codereview.webrtc.org/1972083002 .
Cr-Commit-Position: refs/heads/master@{#13018}
Currently there are two structs that are identical and track extension details:
webrtc::RtpExtension
cricket::RtpHeaderExtension
The use of the structs is mixed in the code to track the extensions being
supported. This results in duplicate definition of
the URI constants and there is code to convert between the two structs.
Clean up to use a single RtpHeader throughout the codebase. The actual location
of RtpHeader may change in future (perhaps to be located in api/). Additionally,
this CL renames some of the constants to clarify Uri and Id use.
BUG= webrtc:5895
Review-Url: https://codereview.webrtc.org/1984983002
Cr-Commit-Position: refs/heads/master@{#12924}
Also set the Configuration parameters in CreateRtpRtcpModules in the same order as the members are declared.
BUG=webrtc:5917
Review-Url: https://codereview.webrtc.org/2011433002
Cr-Commit-Position: refs/heads/master@{#12905}
Remove ViEEncoder::SetNetworkStatus.
Original cl description:
This cl removed ViEEncoder::SetNetworkStatus. Instead the PacedSender will report that frames can not be sent when the network is down and the BitrateController will report an estimated available bandwidth of 0 bps.
Patchset #1 is a pure reland.
Patchset #2 change the bitrate allocator to always return an initial bitrate >0 regardless of the estimates. The observer will be notified though if the network is down.
BUG=webrtc:5687
Review-Url: https://codereview.webrtc.org/1972183004
Cr-Commit-Position: refs/heads/master@{#12743}
This CL contains a few minor changes to names, function signatures and
merges two structs into one.
BUG=5868
Review-Url: https://codereview.webrtc.org/1952923005
Cr-Commit-Position: refs/heads/master@{#12716}
