In order for the VCMTiming object to be correctly updated with decoding timings
when running the WebRTC-NewVideoJitterBuffer experiment the VCMTiming object
has to be available in both the VideoReceiver and the video_coding::FrameBuffer
class. Therefore the VCMTiming object is created in VideoRecieveStream and
then passed to VideoReceiver/video_coding::FrameBuffer as they are constructed.
BUG=webrtc:5514
Review-Url: https://codereview.webrtc.org/2575473004
Cr-Commit-Position: refs/heads/master@{#15638}
This brings QualityScaler much more in line with OveruseFrameDetector.
The two classes are conceptually similar, and should be used in the
same way. The biggest changes in this CL are:
- Quality scaling is now only done in ViEEncoder and not in each
encoder implementation separately.
- QualityScaler now checks the average QP asynchronously, instead of
having to be polled on each frame.
- QualityScaler is no longer responsible for actually scaling the frames,
but has a callback to ViEEncoder that it uses to express it's desire
for lower resolution.
BUG=webrtc:6495
Review-Url: https://codereview.webrtc.org/2398963003
Cr-Commit-Position: refs/heads/master@{#15286}
A recent cl (https://codereview.webrtc.org/2510583002) introduced an
issue where the initial rate allocation (call to VideoBitrateAllocator
and any associated temporal layers) uses framerate = 0 fps. This may
cause issues, including having the rate control in ScreenshareLayers
ramp up too slowly.
This CL make the initial call use VideoCodec.maxFramerate as framerate.
Also expanded unit tests.
BUG=webrtc:6301
Review-Url: https://codereview.webrtc.org/2513383002
Cr-Commit-Position: refs/heads/master@{#15166}
This is yet another reland of https://codereview.webrtc.org/2434073003/
including two fixes:
1. SimulcastRateAllocator did not handle the screenshare settings properly for numSimulcastStreams = 1. Additional test case was added for that.
2. In VideoSender, when rate allocation is updated after setting a new VideoCodec config, only update the state of the EncoderParameters, but don't actually run SetRateAllocation on the encoder itself. This caused some problems upstreams.
Please review only the changes after patch set 1.
Original description:
Extract bitrate allocation of spatial/temporal layers out of codec impl.
This CL makes a number of intervowen changes:
* Add BitrateAllocation struct, that contains a codec independent view
of how the target bitrate is distributed over spatial and temporal
layers.
* Adds the BitrateAllocator interface, which takes a bitrate and frame
rate and produces a BitrateAllocation.
* A default (non layered) implementation is added, and
SimulcastRateAllocator is extended to fully handle VP8 allocation.
This includes capturing TemporalLayer instances created by the
encoder.
* ViEEncoder now owns both the bitrate allocator and the temporal layer
factories for VP8. This allows allocation to happen fully outside of
the encoder implementation.
This refactoring will make it possible for ViEEncoder to signal the
full picture of target bitrates to the RTCP module.
BUG=webrtc:6301
R=stefan@webrtc.org
Review URL: https://codereview.webrtc.org/2510583002 .
Cr-Commit-Position: refs/heads/master@{#15105}
Reason for revert:
Seems to be causing flakiness in perf test:
FullStackTest.ScreenshareSlidesVP8_2TL_LossyNet
Original issue's description:
> Reland of Issue 2434073003: Extract bitrate allocation ...
>
> This is a reland of https://codereview.webrtc.org/2434073003/ including
> some fixes for failing test cases.
>
> Original description:
>
> Extract bitrate allocation of spatial/temporal layers out of codec impl.
>
> This CL makes a number of intervowen changes:
>
> * Add BitrateAllocation struct, that contains a codec independent view
> of how the target bitrate is distributed over spatial and temporal
> layers.
>
> * Adds the BitrateAllocator interface, which takes a bitrate and frame
> rate and produces a BitrateAllocation.
>
> * A default (non layered) implementation is added, and
> SimulcastRateAllocator is extended to fully handle VP8 allocation.
> This includes capturing TemporalLayer instances created by the
> encoder.
>
> * ViEEncoder now owns both the bitrate allocator and the temporal layer
> factories for VP8. This allows allocation to happen fully outside of
> the encoder implementation.
>
> This refactoring will make it possible for ViEEncoder to signal the
> full picture of target bitrates to the RTCP module.
>
> BUG=webrtc:6301
>
> Committed: https://crrev.com/647bf43dcb2fd16fccf276bd94dc4400728bb405
> Cr-Commit-Position: refs/heads/master@{#15023}
TBR=mflodman@webrtc.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG=webrtc:6301
Review-Url: https://codereview.webrtc.org/2491393002
Cr-Commit-Position: refs/heads/master@{#15026}
This is a reland of https://codereview.webrtc.org/2434073003/ including
some fixes for failing test cases.
Original description:
Extract bitrate allocation of spatial/temporal layers out of codec impl.
This CL makes a number of intervowen changes:
* Add BitrateAllocation struct, that contains a codec independent view
of how the target bitrate is distributed over spatial and temporal
layers.
* Adds the BitrateAllocator interface, which takes a bitrate and frame
rate and produces a BitrateAllocation.
* A default (non layered) implementation is added, and
SimulcastRateAllocator is extended to fully handle VP8 allocation.
This includes capturing TemporalLayer instances created by the
encoder.
* ViEEncoder now owns both the bitrate allocator and the temporal layer
factories for VP8. This allows allocation to happen fully outside of
the encoder implementation.
This refactoring will make it possible for ViEEncoder to signal the
full picture of target bitrates to the RTCP module.
BUG=webrtc:6301
Review-Url: https://codereview.webrtc.org/2488833004
Cr-Commit-Position: refs/heads/master@{#15023}
Reason for revert:
Breaks perf tests.
Original issue's description:
> Extract bitrate allocation of spatial/temporal layers out of codec impl.
>
> This CL makes a number of intervowen changes:
>
> * Add BitrateAllocation struct, that contains a codec independent view
> of how the target bitrate is distributed over spatial and temporal
> layers.
>
> * Adds the BitrateAllocator interface, which takes a bitrate and frame
> rate and produces a BitrateAllocation.
>
> * A default (non layered) implementation is added, and
> SimulcastRateAllocator is extended to fully handle VP8 allocation.
> This includes capturing TemporalLayer instances created by the
> encoder.
>
> * ViEEncoder now owns both the bitrate allocator and the temporal layer
> factories for VP8. This allows allocation to happen fully outside of
> the encoder implementation.
>
> This refactoring will make it possible for ViEEncoder to signal the
> full picture of target bitrates to the RTCP module.
>
> BUG=webrtc:6301
>
> Committed: https://crrev.com/8f46c679d24a05b3f08e02c6d91ec9637f34e24f
> Cr-Commit-Position: refs/heads/master@{#14998}
TBR=stefan@webrtc.org,perkj@webrtc.org,mflodman@webrtc.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG=webrtc:6301
Review-Url: https://codereview.webrtc.org/2489843002
Cr-Commit-Position: refs/heads/master@{#15001}
This CL makes a number of intervowen changes:
* Add BitrateAllocation struct, that contains a codec independent view
of how the target bitrate is distributed over spatial and temporal
layers.
* Adds the BitrateAllocator interface, which takes a bitrate and frame
rate and produces a BitrateAllocation.
* A default (non layered) implementation is added, and
SimulcastRateAllocator is extended to fully handle VP8 allocation.
This includes capturing TemporalLayer instances created by the
encoder.
* ViEEncoder now owns both the bitrate allocator and the temporal layer
factories for VP8. This allows allocation to happen fully outside of
the encoder implementation.
This refactoring will make it possible for ViEEncoder to signal the
full picture of target bitrates to the RTCP module.
BUG=webrtc:6301
Review-Url: https://codereview.webrtc.org/2434073003
Cr-Commit-Position: refs/heads/master@{#14998}
This cl is in preparation for https://codereview.webrtc.org/2060403002/ Add task queue to Call.
In the coming cl the video_sender, and i420_buffer_pool will be used on a task queue and therefore SequencedTaskChecker is needed instead of a ThreadChecker.
BUG=webrtc:5687
Review-Url: https://codereview.webrtc.org/2149553002
Cr-Commit-Position: refs/heads/master@{#13474}
When the target bitrate is zero, currently VideoSendStream.Stats.target_media_bitrate_bps show the last set rate before the target was set to zero.
BUG=webrtc::5687 b/29574845
Review-Url: https://codereview.webrtc.org/2122743003
Cr-Commit-Position: refs/heads/master@{#13386}
I'm also removing media_optimization_unittest.cc, since it only tested the
suspension logic and nothing else.
R=pbos@webrtc.org
Review URL: https://codereview.webrtc.org/2119503002 .
Cr-Commit-Position: refs/heads/master@{#13355}
This cl change so that VideoSendStream::Start adds the stream as a BitrateObserver and VideoSendStream::Stop removes the stream as observer.
That also means that start will trigger a VideoEncoder::SetRate call with the most recent bitrate estimate.
VideoSendStream::Stop will trigger a VideoEncoder::SetRate with bitrate = 0.
BUG=webrtc:5687 b/28636240
Review-Url: https://codereview.webrtc.org/2070343002
Cr-Commit-Position: refs/heads/master@{#13192}
This cl split the class MediaOptimization into two parts. One that deals with frame dropping and stats and one new class called ProtectionBitrateCalculator that deals with calculating the needed FEC parameters and how much of the estimated network bitrate that can be used by an encoder
Note that the logic of how FEC and the needed bitrates is not changed.
BUG=webrtc:5687
R=asapersson@webrtc.org, stefan@webrtc.org
Review URL: https://codereview.webrtc.org/1972083002 .
Cr-Commit-Position: refs/heads/master@{#13018}
SSRC knowledge is contained withing VideoSendStream. That also means that debug recording is moved to VideoSendStream.
I think that make sence since that allows debug recording with external encoder implementations one day.
BUG=webrtc:5687
Review-Url: https://codereview.webrtc.org/1936503002
Cr-Commit-Position: refs/heads/master@{#12632}
And move encoder name cb to VCMSendStatisticsCallback.
BUG=webrtc:5687
Review-Url: https://codereview.webrtc.org/1900193004
Cr-Commit-Position: refs/heads/master@{#12596}
ViEEncoder doesn't need a full VideoCodingModule since it only uses the
sender side either way.
BUG=webrtc:3608,webrtc:5687
R=perkj@webrtc.org
Review URL: https://codereview.webrtc.org/1904983002 .
Cr-Commit-Position: refs/heads/master@{#12456}
Reason for revert:
A fix is being prepared downstream so this can now go in.
Original issue's description:
> Revert of Deprecate VCMPacketizationCallback::SendData and use EncodedImageCallback instead. (patchset #5 id:80001 of https://codereview.webrtc.org/1897233002/ )
>
> Reason for revert:
> API changes broke downstream.
>
> Original issue's description:
> > Deprecate VCMPacketizationCallback::SendData and use EncodedImageCallback instead.
> > EncodedImageCallback is used by all encoder implementations and seems to be what we should try to use in the transport.
> > EncodedImageCallback can of course be cleaned up in the future.
> >
> > This moves creation of RTPVideoHeader from the GenericEncoder to the PayLoadRouter.
> >
> > BUG=webrtc::5687
> >
> > Committed: https://crrev.com/f5d55aaecdc39e9cc66eb6e87614f04afe28f6eb
> > Cr-Commit-Position: refs/heads/master@{#12436}
>
> TBR=stefan@webrtc.org,pbos@webrtc.org,perkj@webrtc.org
> # Skipping CQ checks because original CL landed less than 1 days ago.
> NOPRESUBMIT=true
> NOTREECHECKS=true
> NOTRY=true
> BUG=webrtc:5687
>
> Committed: https://crrev.com/a261e6136655af33f283eda8e60a6dd93dd746a4
> Cr-Commit-Position: refs/heads/master@{#12441}
TBR=stefan@webrtc.org,pbos@webrtc.org,perkj@webrtc.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG=webrtc:5687
Review URL: https://codereview.webrtc.org/1905583002
Cr-Commit-Position: refs/heads/master@{#12442}
Reason for revert:
API changes broke downstream.
Original issue's description:
> Deprecate VCMPacketizationCallback::SendData and use EncodedImageCallback instead.
> EncodedImageCallback is used by all encoder implementations and seems to be what we should try to use in the transport.
> EncodedImageCallback can of course be cleaned up in the future.
>
> This moves creation of RTPVideoHeader from the GenericEncoder to the PayLoadRouter.
>
> BUG=webrtc::5687
>
> Committed: https://crrev.com/f5d55aaecdc39e9cc66eb6e87614f04afe28f6eb
> Cr-Commit-Position: refs/heads/master@{#12436}
TBR=stefan@webrtc.org,pbos@webrtc.org,perkj@webrtc.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG=webrtc::5687
Review URL: https://codereview.webrtc.org/1903193002
Cr-Commit-Position: refs/heads/master@{#12441}
EncodedImageCallback is used by all encoder implementations and seems to be what we should try to use in the transport.
EncodedImageCallback can of course be cleaned up in the future.
This moves creation of RTPVideoHeader from the GenericEncoder to the PayLoadRouter.
BUG=webrtc::5687
Review URL: https://codereview.webrtc.org/1897233002
Cr-Commit-Position: refs/heads/master@{#12436}
Also refactor GenericEncoder to use these file writers, and remove use
of preprocessor to enable file writing.
BUG=
Review URL: https://codereview.webrtc.org/1853813002
Cr-Commit-Position: refs/heads/master@{#12372}
- First audio RTP packet sent / received
- First RTP packet of the first video frame sent / received
- Last RTP packet of the first video frame sent / received
These timestamps should make it easier to measure how fast the call
becomes established from the user's perspective.
Review URL: https://codereview.webrtc.org/1765443002
Cr-Commit-Position: refs/heads/master@{#12287}
Testing the nack module by implementing it into the current jitter buffer
under the experiment WebRTC-NewVideoJitterBuffer.
BUG=webrtc:5514
Review URL: https://codereview.webrtc.org/1778503002
Cr-Commit-Position: refs/heads/master@{#11969}
Reason for revert:
Revert breaks other uses, a fix will be rolled into Chromium instead.
Original issue's description:
> Revert of Remove ignored return code from modules. (patchset #3 id:40001 of https://codereview.webrtc.org/1703833002/ )
>
> Reason for revert:
> Breaks Chromium.
>
> Original issue's description:
> > Remove ignored return code from modules.
> >
> > ModuleProcessImpl doesn't act on return codes and having them around is
> > confusing (it's unclear what an error return code here would do even).
> >
> > BUG=
> > R=tommi@webrtc.org
> >
> > Committed: f14c47a58c
>
> TBR=tommi@webrtc.org,pbos@webrtc.org
> # Skipping CQ checks because original CL landed less than 1 days ago.
> NOPRESUBMIT=true
> NOTREECHECKS=true
> NOTRY=true
> BUG=
>
> Committed: https://crrev.com/da33a8a2a22f6d19ba2a8cce963beafbdbaa8fd8
> Cr-Commit-Position: refs/heads/master@{#11761}
TBR=tommi@webrtc.org,torbjorng@webrtc.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG=
Review URL: https://codereview.webrtc.org/1737013002
Cr-Commit-Position: refs/heads/master@{#11762}
Reason for revert:
Breaks Chromium.
Original issue's description:
> Remove ignored return code from modules.
>
> ModuleProcessImpl doesn't act on return codes and having them around is
> confusing (it's unclear what an error return code here would do even).
>
> BUG=
> R=tommi@webrtc.org
>
> Committed: f14c47a58cTBR=tommi@webrtc.org,pbos@webrtc.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG=
Review URL: https://codereview.webrtc.org/1736663004
Cr-Commit-Position: refs/heads/master@{#11761}
ModuleProcessImpl doesn't act on return codes and having them around is
confusing (it's unclear what an error return code here would do even).
BUG=
R=tommi@webrtc.org
Review URL: https://codereview.webrtc.org/1703833002 .
Cr-Commit-Position: refs/heads/master@{#11747}
Permits measuring encoding time even when performed on another thread,
typically for hardware encoding, instead of assuming that encoding is
blocking the calling thread.
Permitted encoding time is increased for hardware encoders since they
can be timed to keep 30fps, for instance, without indicating overload.
Merges EncodingTimeObserver into EncodedFrameObserver to have one post-encode
callback.
BUG=webrtc:5042, webrtc:5132
R=asapersson@webrtc.org, mflodman@webrtc.org
Review URL: https://codereview.webrtc.org/1569853002 .
Cr-Commit-Position: refs/heads/master@{#11499}
The purpose is so that a decoder (Android) that only have a limited number of output buffers can make sure that decoding is done just before the frame is needed.
Removed unused iSupportsRenderTiming and the settings structs since it was not used.
Added VCMReceiver::FrameForDecoding unit test for the case when PreferDecodeLate is set.
Note that this does not change the current behaviour. We actually currently always decode frames late. This cl is to make sure the behaviour is kept for Android, if the default behaviour is changed.
Review URL: https://codereview.webrtc.org/1428293003
Cr-Commit-Position: refs/heads/master@{#10974}
Also adds a RTC_CHECK in VideoReceiveStream that verifies that decoders
aren't null, since this will attempt to deregister a codec which would
previously fail with an obscure stack trace not indicating what actually
was wrong.
BUG=webrtc:5249
R=stefan@webrtc.org
Review URL: https://codereview.webrtc.org/1479793002 .
Cr-Commit-Position: refs/heads/master@{#10821}
This makes it clearer this code not meant to be used as an API.
I could not find any use of this in downstream code.
BUG=webrtc:5095
TESTED=git cl try -c --bot=android_compile_rel --bot=linux_compile_rel --bot=win_compile_rel --bot=mac_compile_rel --bot=ios_rel --bot=linux_gn_rel --bot=win_x64_gn_rel --bot=mac_x64_gn_rel --bot=android_gn_rel -m tryserver.webrtc
R=stefan@webrtc.orgTBR=magjed@webrtc.org
Review URL: https://codereview.webrtc.org/1440873005 .
Cr-Commit-Position: refs/heads/master@{#10699}
The main purpose was the interface-> include rename, but other files
were also moved, eliminating the "main" dir.
To avoid breaking downstream, the "interface" directories were copied
into a new "video_coding/include" dir. The old headers got pragma
warnings added about deprecation (a very short deprecation since I plan
to remove them as soon downstream is updated).
Other files also moved:
video_coding/main/source -> video_coding
video_coding/main/test -> video_coding/test
BUG=webrtc:5095
TESTED=Passing compile-trybots with --clobber flag:
git cl try --clobber --bot=win_compile_rel --bot=linux_compile_rel --bot=android_compile_rel --bot=mac_compile_rel --bot=ios_rel --bot=linux_gn_rel --bot=win_x64_gn_rel --bot=mac_x64_gn_rel --bot=android_gn_rel -m tryserver.webrtc
R=stefan@webrtc.org, tommi@webrtc.org
Review URL: https://codereview.webrtc.org/1417283007 .
Cr-Commit-Position: refs/heads/master@{#10694}
