This CL is part of the Call-Level Adaptation Processing design doc:
https://docs.google.com/document/d/1ZyC26yOCknrrcYa839ZWLxD6o6Gig5A3lVTh4E41074/edit?usp=sharing
This gets to the heart of unblocking call-level adaptation, largely
made possible due to the previous CLs in the chain.
The parts of the code that are responsible for responding to resource
usage signals, obtaining adaptations and applying them are moved to
ResourceAdaptationProcessor in call/adaptation/.
The parts of the code that are responsible for managing
VideoStreamEncoder-specific resources stay inside the
VideoStreamEncoderResourceManager class in video/adaptation/.
After this CL lands it should soon be possible to move the Processor
over to a separate task queue and let the Manager stay on the encoder
queue if PostTasks are added for communication between the two objects.
Bug: webrtc:11172
Change-Id: Ifa212467b4afd16e7ebfb9adfe17d2dca1cb7d67
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/173021
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Evan Shrubsole <eshr@google.com>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#31105}
This CL is part of the Call-Level Adaptation Processing design doc:
https://docs.google.com/document/d/1ZyC26yOCknrrcYa839ZWLxD6o6Gig5A3lVTh4E41074/edit?usp=sharing
By pushing VideoAdaptationCounters updates on VideoSourceRestrictions
changes, alongside the Resource* that triggered the adaptation, we are
able to update |active_counts_| without an explicit dependency on the
VideoStreamAdapter. This allows a future CL to split up "processor"
logic from "video stream encoder resource and active counts" logic,
which will ultimately be necessary in order to do processing on a
"processing queue" and encoder and stats logic on the "encoder queue".
If the restrictions got cleared by an API call
(ResetVideoSourceRestrictions() or SetDegradationPreference()) we pass
null as the "reason_resource". This allows is to clear the
active_counts_, and the code that invokes
OnVideoSourceRestrictionsUpdated() does not have to be aware of
active_counts_ (needed to split the processor module in two).
Bug: webrtc:11172
Change-Id: Icab6d5121c0ebd27d2a00f1bffc8191f8f05f562
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/173000
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Evan Shrubsole <eshr@google.com>
Cr-Commit-Position: refs/heads/master@{#31103}
This CL is part of the Call-Level Adaptation Processing design doc:
https://docs.google.com/document/d/1ZyC26yOCknrrcYa839ZWLxD6o6Gig5A3lVTh4E41074/edit?usp=sharing
The VideoStreamAdapter is currently responsible for aborting and not
providing adaptations if we are bitrate constrained
(kIsBitrateConstrained). Whether or not we are bitrate constrained is
clearly a resource question and should be phrased as such. By moving
this logic to Resource::IsAdaptationUpAllowed(), the VideoStreamAdapter
can continue to be thread-agnostic when a future CL introduces a
"processing queue", and the VideoStreamAdapter can be simplified: it
returns Adaptations even if we are constrained (but we refuse to Apply
them any resource rejects it).
This CL adds new Resource classes as inner classes of
ResourceAdaptationProcessor that take on the responsibility of
kIsBitrateConstrained logic:
PreventIncreaseResolutionDueToBitrateResource and
PreventAdaptUpInBalancedResource.
A third class, PreventAdaptUpDueToActiveCounts, also allows us to move
adaptation-aborting logic. This piece of code appears to be about not
adapting up if we’re already at the highest setting, which would be
VideoStreamAdapter responsibility (covered by
Adaptation::Status::kLimitReached), but it is actually more complicated
than that: the active_counts_ care about "reason", so it is really about
"is this resource type OK with you adapting up?". We should probably
rewrite this code in the future, but for now it is moved to an inner
class of ResourceAdaptationProcessor.
Other misc changes:
- ApplyDegradationPreference is moved to video_stream_adapter.[h/cc]
and renamed "Filter".
- OnResourceOveruse/Underuse now use Resource* as the reason instead of
AdaptReason. In a future CL, the processor will be split into a
"processor" part and a "video stream encoder resource manager" part.
Only the manager needs to know about AdaptReason since this is only
used for |active_counts_| and we want to get rid of it as much as
possible as it is not future-proof.
Bug: webrtc:11172
Change-Id: I2eba9ec3d717f7024c451aeb14635fe759551318
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/172930
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Evan Shrubsole <eshr@google.com>
Cr-Commit-Position: refs/heads/master@{#31099}
This CL is part of the Call-Level Adaptation Processing design doc:
https://docs.google.com/document/d/1ZyC26yOCknrrcYa839ZWLxD6o6Gig5A3lVTh4E41074/edit?usp=sharing
The "input state" of a VideoStream, needed for adaptation and
decision-making, are: source resolution and frame rate, codec type and
min pixels per frame (based on encoder scaling settings). These values
are modified on the encoder queue of the VideoStreamEncoder.
But in order to unblock call-level adaptation processing, where
adaptation and decision making happens off the encoder queue, a snapshot
of the input states need to be available at point of processing:
introducing the VideoStreamInputState.
In this CL, the VideoStreamInputStateProvider is added to provide input
state snapshots across threads based on input from VideoStreamEncoder
and VideoStreamEncoderObserver.
The input state's HasInputFrameSizeAndFramesPerSecond() can now be
DCHECKed inside the VideoStreamAdapter in favor of having less
Adaptation::Status codes. Whether input is "sufficient" for adaptation
is now the responsibility of the Processor. (Goal: adapter is purely a
Adaptation generator and apply-er.)
Somewhat tangental, this CL also deletes VideoStreamEncoder-specific
methods from ResourceAdaptationProcessorInterface making them an
implementation detail of ResourceAdaptationProcessor. In a future CL,
the "processor" will be split up into a "processor" part and a "video
stream encoder resource manager" part - more on that later.
Bug: webrtc:11172
Change-Id: Id9b158f569db0140b75360aaf0f7e2e28fb924f4
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/172928
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Evan Shrubsole <eshr@google.com>
Cr-Commit-Position: refs/heads/master@{#31098}
This CL is part of the Call-Level Adaptation Processing design doc:
https://docs.google.com/document/d/1ZyC26yOCknrrcYa839ZWLxD6o6Gig5A3lVTh4E41074/edit?usp=sharing
The VideoStreamAdapter is responsible for generating adaptation
suggestions according to its DegradationPreference. Today there is one
DegradationPreference that you set, but internally the preference it
uses to make decisions is EffectiveDegradationPreference() which
reinterprets “balanced” as “maintain-resolution” if screenshare is used.
By moving the “effective” logic to the ResourceAdaptationProcessor, the
VideoStreamAdapter will not need to know about the type of track, and
the responsibility of the adapter is minimized. The “effective” logic
is non-standard and something we want to get rid of - until then, it
should be the responsibility of the processor to configure the adapter
to use the appropriate strategy, rather than for the adapter to know
about more states of the system than it needs to.
Future CLs will further minimize what the adapter needs to know, moving
"decision-making" logic to the Processor and "is adapt up allowed?"
logic to the Resources.
By removing the VideoInputMode enum the VideoStreamAdapter does not
know if we have input which has to be checked externally. Input
handling is followed-up on in the next CL.
Bug: webrtc:11172
Change-Id: I37ec9e7392f835cf8fef9829a2c945183f0e9b65
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/172927
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Evan Shrubsole <eshr@google.com>
Cr-Commit-Position: refs/heads/master@{#31096}
Today when the pacing debt is cleared, we blindly ask for 50 bytes of
padding, which is above a static magic number for RTX payload padding.
Instead, we should adjust the target size based on the current padding
rate. The old pacer sort-of does this, it allows the budget to grow up
to one process interval (usually 5ms).
This CL makes the dynamic pacer also use a duration as target, by
default 5ms to match old pacer but with a trial to allow tweaking it.
This will be important for good behavior due to
https://bugs.chromium.org/p/webrtc/issues/detail?id=11508
Bug: webrtc:10809
Change-Id: I9c14acc5730c6e2e0d7821adf5fb058b8d5487c6
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/173687
Commit-Queue: Erik Språng <sprang@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#31091}
This was introduced on trial but turned out to perform badly for WebRTC
purposes and never used in production.
Bug: webrtc:9883
Change-Id: Ib72acddf4d90fc9ac042084dddf526c04661f290
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/173680
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Sebastian Jansson <srte@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#31085}
Replaces this with 2 methods instead, adding clarity.
ClearAdaptationStats
- Resets the adaptations statistics to 0. This is done,
when the degredation is reset, for example when the preference
is changed to/from BALANCED.
UpdateAdaptationMaskingSettings
- Updates the settings for adaptation statistics reporting.
This way we don't report quality adaptations if quality scaling
is not enabled (same for resolution/fps scaling).
The adaptation counting inside the SendStatisticsProxy is
now done in a struct that counts the totals, and then masks
out these counts based on the adaptation settings. The
MaskedAdaptationSteps uses optionals to hide the values we
shoudn't report, while the AdaptationSteps always hold the real
totals.
All tests have been updated to use the Reset/Clear method as needed.
Now that AdaptationCounters and AdaptSteps use the same structure,
AdaptationCounters was moved to api/video and replaces AdaptSteps.
The AdaptReason enum is also redundant now, and will be removed
in a follow-up CL.
R=hbos@webrtc.org
Bug: webrtc:11392
Change-Id: Iaed6488581325d341a056b5bbf76a01c19d6c282
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/171685
Reviewed-by: Niels Moller <nisse@webrtc.org>
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Evan Shrubsole <eshr@google.com>
Cr-Commit-Position: refs/heads/master@{#31083}
We have seen crashes originating from derefencing nullptrs in this code,
for unknown reasons. This CL adds null checks to protect against this.
The stacktraces will be missing or truncated when this happens.
Bug: b/147338449
Change-Id: Ieb006f0f8dec4f9621e4df2e2c1a9641f086df86
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/173593
Reviewed-by: Sami Kalliomäki <sakal@webrtc.org>
Commit-Queue: Magnus Jedvert <magjed@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#31079}
https://webrtc-review.googlesource.com/c/src/+/172847
------------ original description --------------
Preparation for ReceiveStatisticsProxy lock reduction.
Update tests to call VideoReceiveStream::GetStats() in the same or at
least similar way it gets called in production (construction thread,
same TQ/thread).
Mapped out threads and context for ReceiveStatisticsProxy,
VideoQualityObserver and VideoReceiveStream. Added
follow-up TODOs for webrtc:11489.
One functional change in ReceiveStatisticsProxy is that when sender
side RtcpPacketTypesCounterUpdated calls are made, the counter is
updated asynchronously since the sender calls the method on a different
thread than the receiver.
Make CallClient::SendTask public to allow tests to run tasks in the
right context. CallClient already does this internally for GetStats.
Remove 10 sec sleep in StopSendingKeyframeRequestsForInactiveStream.
Bug: webrtc:11489
Change-Id: I491e13344b9fa714de0741dd927d907de7e39e83
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/173583
Commit-Queue: Tommi <tommi@webrtc.org>
Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#31077}
This change fixes declarations that have initial values but are
technically not definitions by marking them constexpr (which counts as a
definition).
Bug: None
Change-Id: Icbecf8d83faffa83b9f7e1ffe4d6ef3a3f0b0c2a
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/173587
Reviewed-by: Karl Wiberg <kwiberg@webrtc.org>
Commit-Queue: Mirko Bonadei <mbonadei@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#31073}
This version of the method is deprecated in favor of the one that takes
an url parameter.
Bug: webrtc:10198
Change-Id: I7614b9cb98217663b0e2fbf1785ae1fb1484beec
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/173333
Commit-Queue: Mirko Bonadei <mbonadei@webrtc.org>
Reviewed-by: Harald Alvestrand <hta@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#31068}
This is a reland of 12e2d4ddb235da6ec7a5c1c3a83ac33d394920b0
Original change's description:
> APM: Remove the usage of AudioFrame in the AudioProcessing interface
>
> This CL removes the AudioFrame-based APIs from the AudioProcessing
> interface.
>
> Bug: webrtc:5298
> Change-Id: Iab470b26b10e06dcf29c543851ae0085bc5b66f0
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/172939
> Reviewed-by: Sam Zackrisson <saza@webrtc.org>
> Commit-Queue: Per Åhgren <peah@webrtc.org>
> Cr-Commit-Position: refs/heads/master@{#31016}
Bug: webrtc:5298
Change-Id: I70e6d59afc3716ee6109d8b9dc384abc71c93624
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/173476
Reviewed-by: Sam Zackrisson <saza@webrtc.org>
Commit-Queue: Per Åhgren <peah@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#31066}
Since there is no plan to follow-up on this, this CL removes the
field trial and the conditional logic based on it.
Bug: webrtc:11503, webrtc:10282
Change-Id: Iaf005eba6af0e23ea50456d75c5c53f37d488f7d
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/173477
Reviewed-by: Björn Terelius <terelius@webrtc.org>
Reviewed-by: Jonas Oreland <jonaso@webrtc.org>
Commit-Queue: Mirko Bonadei <mbonadei@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#31062}
This is more logical way to remove inactive lower layers.
Current way is to notify the encoder that the layer is inactive,
then renumber layers at the packatization level.
This Cl will allow to simplify libvpx vp9 encoder, svcRateAllocator and
vp9 packetizer.
Bug: webrtc:11319
Change-Id: Idf0bb30b729f5ecc97e31454b32934546b681aa2
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/173182
Commit-Queue: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#31058}
These static functions were marked as deprecated and since they
are not used this CL just removes them.
Bug: webrtc:10198
Change-Id: I4872e31701543c988fe71ab4e0b32bd73ff07753
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/173467
Commit-Queue: Mirko Bonadei <mbonadei@webrtc.org>
Reviewed-by: Karl Wiberg <kwiberg@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#31057}
