The EffectiveDegradationPreference() exposed an "implementation detail"
of the VideoStreamAdapter - how degradation preference may be modified.
By changing the return value of ApplyAdaptationTarget() this dependency
could be removed. We still have a TODO to get rid of the
ResourceListenerResponse enum, but that is QualityScaler related work.
This CL does the following:
- Module's GetAdaptUpTarget/GetAdaptDownTarget/ApplyAdaptationTarget
methods are removed in favor if invoking the VideoStreamAdapter's
version of these methods directly.
- Removing the EffectiveDegradationPreference() usage in
OveruseFrameDetectorResourceAdaptationModule meant moving that usage
to VideoStreamAdapter.
- MinPixelsPerFrame() is moved to VideoStreamAdapter; this is "can
adapt?" logic, i.e. the adapter's responsibility.
Bug: webrtc:11393
Change-Id: I75091ce97093bfa48a6d883492de30ed4b004492
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/169859
Reviewed-by: Evan Shrubsole <eshr@google.com>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30714}
This CL moves GetAdaptUpTarget(), GetAdaptDownTarget() and
ApplyAdaptationTarget() - and related code - to the VideoStreamAdapter.
This includes pieces related to calculating how to adapt, including:
- DegradationPreference
- BalancedDegradationPreference
- AdaptationRequest and last_adaptation_request_
- CanAdaptUpResolution()
The VideoStreamAdapter's interface has changed: VideoSourceRestrictor
methods are now hidden in favor of methods exposing AdaptationTarget.
This CL also does some misc moves:
- GetEncoderBitrateLimits is moved and renamed to
VideoEncoder::EncoderInfo::GetEncoderBitrateLimitsForResolution.
- EncoderSettings moved to a separate file.
// For api/video_codecs/video_encoder.[cc/h] changes, which is the
// moving of a function.
TBR=sprang@webrtc.org
Bug: webrtc:11393
Change-Id: Ie6bd8ef644ce927d7eca6ab90a0a7bcace682f3c
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/169842
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Evan Shrubsole <eshr@google.com>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30708}
This CL simply moves the VideoSourceRestrictor from being an inner class
of OveruseFrameDetectorResourceAdaptationModule to a new class,
VideoStreamAdapter.
In follow-up CLs, the responsibility of determining what the next step
for adapting up or down should also be moved to the VideoStreamAdapter.
The end-goal is that the VideoStreamAdapter takes care of "can adapt?"
and "do adapt!" type of logic so that a multi-stream aware adaptation
module can decide which stream (adapter) to adapt, and the adapter can
take care of the nitty gritty details of doing so.
In this CL the "can?"/"do!" part is realized but not the logic for
determining what the next step up or down is, and the class interface
needs improvement.
This CL also sets up the video/adaptation/ subdirectory and moves the
AdaptationCounters class here. Other adaptation-related classes (e.g.
the module and its resources) should move into this directory as well
in the future.
Bug: webrtc:11393
Change-Id: I2c12c1281eca854c62791abb65f0aca47a119726
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/169542
Reviewed-by: Evan Shrubsole <eshr@google.com>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Henrik Andreassson <henrika@webrtc.org>
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30705}
There is still a counter for the active counts for the
scaling, but these will be removed at a later date.
BUG=webrtc:11392
Change-Id: Ie9bcf3f744a0bbac601f0da61197f4bac1e9f879
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/169447
Reviewed-by: Åsa Persson <asapersson@webrtc.org>
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Commit-Queue: Evan Shrubsole <eshr@google.com>
Cr-Commit-Position: refs/heads/master@{#30701}
When selecting all streams there was an index out of bounds
checking the selected temporal layer, which is -1 so was irrelevant.
My fix is to prevent selecting a temporal layer and all streams
at the same time.
Bug: webrtc:11402
Change-Id: I0641b926cba35878945b866f2c59b4b0281f0852
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/169720
Reviewed-by: Stefan Holmer <stefan@webrtc.org>
Commit-Queue: Evan Shrubsole <eshr@google.com>
Cr-Commit-Position: refs/heads/master@{#30679}
This CL takes us one step closer to being able to evaluate alternative
possible adaptation targets (e.g. multi-stream adaptation) by exposing
the target separately from applying it.
This is a refactoring of OnResourceUnderuse() and OnResourceOveruse().
Prior to this CL, the target resolution or frame rate was calculated
inside these methods and applied if possible. This CLs makes these two
steps (calculating a usable target + applying it) separate methods.
After this CL, the target is expressed as AdaptationTarget and is
calculated and returned by GetAdaptUpTarget() and GetAdaptDownTarget().
The target is only returned if it can be applied - and CanAdaptUp() +
CanAdaptDown() are merged with these methods.
Applying the target happens at ApplyAdaptationTarget().
Bug: webrtc:11222
Change-Id: I8e488be1d1590c23848db816d49a7738562e176d
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/169100
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Evan Shrubsole <eshr@google.com>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30643}
The fact that they weren't const is probably a remenant of the good ol'
days this class being multi-threaded and having to acquire mutexes. Now
they can properly be made const.
In order to make GetConstAdaptCounter() const, this CL makes sure a
cleared adapt_counters_ map contains mappings for all degradation
preferences to default-constructed AdaptCounters. Previously, if the
mapping was missing it was implicitly inserted by
GetConstAdaptCounter(). Now it can DCHECK that mappings always exists
instead, and it always has something to return.
Bug: webrtc:11222
Change-Id: If33227fe6572eb1d6cc6b9f851d6d174d035c110
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/168953
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Evan Shrubsole <eshr@google.com>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30611}
* This replaces the video stream methods for forcing adaptation
with a mock resource that triggers overuse.
* Resources can now be injected to the Module using the AddResource
function.
* Resources now have tests for adding and removing callbacks.
* Quality/EncoderUse% resources are tracked in the Resource list of
the adaptation module.
* The adaptation module ties all resources to a reason to keep stats
working as expected.
BUG=webrtc:11377
Change-Id: I1f5902f7416dc41b4915c0072e6f0da2bb3bb2b7
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/168948
Commit-Queue: Evan Shrubsole <eshr@google.com>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30610}
Today OnResourceOveruse() and OnResourceUnderuse() implicitly checks
preconditions and if they pass calculate the next target, and if those
are usable it applies them to the VideoSourceRestrictions. These are two
big "MaybeAdapt" methods.
This CL takes us one step closer to "GetNextTarget", "CanApplyTarget?"
and "DoApplyTarget!"-logic, which will allow us to more easily evaluate
a multitude of alternative configurations and decide which one to pick
(e.g. multi-stream adaptation).
But it does not take us all the way there. In this CL we have:
- CanAdaptUp, CanAdaptDown: This covers *most* of the preconditions.
- OnResourceUnderuse, OnResourceOveruse: This aborts if CanAdapt returns
false. If they pass, we calculate the next target and maybe-adapt it.
This is roughly outlined in document still in draft:
https://docs.google.com/document/d/1YMg-AycFZR1DS6hEav9OzJ3hqxFil09qPhlTAgQrU1g/edit?usp=sharing.
A future CL should make the target more explicit and we should know if
the target can be applied before we even try.
Bug: webrtc:11222
Change-Id: If18d9572884aa6ba2350e4670a1516da5835cc98
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/168721
Reviewed-by: Evan Shrubsole <eshr@google.com>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30605}
Adapting up or down is currently a "Maybe Adapt" method. In the future
we will want to be able to decide which stream to adapt, and as such we
need to be able to tell if a stream is able to do so.
This takes us one step in that direction, by refactoring
OveruseFrameDetectorResourceAdaptationModule::VideoSourceRestrictor's
methods to follow a simple pattern:
- What is the next step?
GetHigherFrameRateThan, GetLowerFrameRateThan,
GetHigherResolutionThan, GetLowerResolutionThan
- Can we adapt?
CanIncreaseFrameRate, CanDecreaseFrameRate,
CanIncreaseResolution, CanDecreaseResolution
- Do adapt!
IncreaseFrameRateTo, DecreaseFrameRateTo,
IncreaseResolutionTo, DecreaseResolutionTo
Hopefully this makes the code easier to follow.
This CL changes the "Request Higher/Lower" methods to take the target
as input instead of implicitly calculating the target from the current
input resolution or frame rate.
Bug: webrtc:11222
Change-Id: If625834e921a24a872145105f5d553fb8f9f1795
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/168966
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Evan Shrubsole <eshr@google.com>
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30600}
This reverts commit 93d9ae8a17f2e7b90641cbac28e740afc67d383a.
Reason for revert: Perf regression.
Original change's description:
> Remove ResourceAdaptationModule::OnMaybeEncodeFrame
>
> We can react just as well at OnEncodeVideoFrame, which is the same
> behaviour except after checking if the Encoder is paused and the frame
> dropper.
>
> For the initial frame drop, the frame dropper is irrelevant as the frame
> can not be dropped until we are accepting frames. If we didn't drop the
> frame, the encoder can't be paused as the data rate
> is over 0.
>
> For the quality rampup experiment, similar for encoder paused - we can't
> rampup if we are paused anyways since the data rate needs to be non-zero.
> If we are dropping frames we likely don't want to do quality rampup
> anyways.
>
> Bug: webrtc:11222
> Change-Id: Ie3e09d9d8d509dc17ba7a1443cf4747f61c04f6a
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/168601
> Reviewed-by: Henrik Boström <hbos@webrtc.org>
> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
> Commit-Queue: Evan Shrubsole <eshr@google.com>
> Cr-Commit-Position: refs/heads/master@{#30539}
TBR=ilnik@webrtc.org,hbos@webrtc.org,eshr@google.com
# Not skipping CQ checks because original CL landed > 1 day ago.
No-Try: True
Bug: webrtc:11222
Change-Id: Ifb2fc74eb7572568fb0ee1b53a09e4180f87b30c
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/168880
Commit-Queue: Mirko Bonadei <mbonadei@webrtc.org>
Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30568}
The change ships GenericDescriptor00 and authentication by default,
but doesn't expose it by default, and makes WebRTC respond to
offers carrying it.
The change adds a unit test for the new semantics.
Tests well in munge-sdp. Frame marking replaced by
http://www.webrtc.org/experiments/rtp-hdrext/generic-frame-descriptor-00
in the offer results in an answer containing the
extension as first entry.
Bug: webrtc:11367
Change-Id: I0ef91b7d4096d949c3d547ece7d6c4d39aa241da
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/168661
Reviewed-by: Magnus Flodman <mflodman@webrtc.org>
Reviewed-by: Danil Chapovalov <danilchap@webrtc.org>
Commit-Queue: Markus Handell <handellm@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30542}
We can react just as well at OnEncodeVideoFrame, which is the same
behaviour except after checking if the Encoder is paused and the frame
dropper.
For the initial frame drop, the frame dropper is irrelevant as the frame
can not be dropped until we are accepting frames. If we didn't drop the
frame, the encoder can't be paused as the data rate
is over 0.
For the quality rampup experiment, similar for encoder paused - we can't
rampup if we are paused anyways since the data rate needs to be non-zero.
If we are dropping frames we likely don't want to do quality rampup
anyways.
Bug: webrtc:11222
Change-Id: Ie3e09d9d8d509dc17ba7a1443cf4747f61c04f6a
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/168601
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Commit-Queue: Evan Shrubsole <eshr@google.com>
Cr-Commit-Position: refs/heads/master@{#30539}
It was used only for the frame decryptor.
Decryptor needs only raw representation that it can recreate
in a way compatible with the new version of the descriptor.
This relands commit abf73de8eae90e9ac7e88ce1d52728e8102e824f.
with adjustments.
Change-Id: I935977179bef31d8e1023964b967658e9a7db92d
Bug: webrtc:10342
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/168489
Reviewed-by: Sam Zackrisson <saza@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Commit-Queue: Danil Chapovalov <danilchap@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30532}
This reverts commit abf73de8eae90e9ac7e88ce1d52728e8102e824f.
Reason for revert: breaks downstream tests
Original change's description:
> Do not propagate generic descriptor on receiving frame
>
> It was used only for the frame decryptor.
> Decryptor needs only raw representation that it can recreate
> in a way compatible with the new version of the descriptor.
>
> Bug: webrtc:10342
> Change-Id: Ie098235ebb87c6f5e2af42d0022d2365cd6bfa29
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/166163
> Reviewed-by: Philip Eliasson <philipel@webrtc.org>
> Reviewed-by: Erik Språng <sprang@webrtc.org>
> Commit-Queue: Danil Chapovalov <danilchap@webrtc.org>
> Cr-Commit-Position: refs/heads/master@{#30501}
TBR=danilchap@webrtc.org,sprang@webrtc.org,philipel@webrtc.org
Change-Id: I6634df06ee75aa8cdfda614994ab11f7a5845c70
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: webrtc:10342
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/168488
Reviewed-by: Danil Chapovalov <danilchap@webrtc.org>
Commit-Queue: Danil Chapovalov <danilchap@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30502}
It was used only for the frame decryptor.
Decryptor needs only raw representation that it can recreate
in a way compatible with the new version of the descriptor.
Bug: webrtc:10342
Change-Id: Ie098235ebb87c6f5e2af42d0022d2365cd6bfa29
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/166163
Reviewed-by: Philip Eliasson <philipel@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Danil Chapovalov <danilchap@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30501}
This is a subset of the module's behavior and accounts for 6 of the
member variables of the OveruseFrameDetectorResourceAdaptationModule.
Isolating this behavior to an inner class makes the module slightly less
convoluted.
Bug: webrtc:11222
Change-Id: Ibb5442afb03a1ee850da590b83cd5afbbb14783d
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/168309
Reviewed-by: Evan Shrubsole <eshr@google.com>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30500}
The EncoderSelectorInterface is meant to replace the "WebRTC-NetworkCondition-EncoderSwitch" field trial, so the field trial will be ignored if an EncoderSelectorInterface object has been injected.
Bug: webrtc:11341
Change-Id: I5371fac9c9ad8e38223a81dd1e7bfefb2bb458cb
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/168193
Commit-Queue: Philip Eliasson <philipel@webrtc.org>
Reviewed-by: Niels Moller <nisse@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Reviewed-by: Rasmus Brandt <brandtr@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30490}
With current congestion window pushback, when congestion window is filling up, it will reduce bitrate directly and encoder may reduce encode quality, resolution, or framerate to adapt to the allocated bitrate, the behavior is depending on the degradation preference.
This change enable congestion window to only drop frames to reduce bitrate (when needed) instead of reduce general bitrate allocation.
Bug: webrtc:11334
Change-Id: I9cf5c20a0858c4d07d006942abe72aa5e1f7cb38
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/168059
Commit-Queue: Ying Wang <yinwa@webrtc.org>
Reviewed-by: Karl Wiberg <kwiberg@webrtc.org>
Reviewed-by: Åsa Persson <asapersson@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30483}
