This replaces references to the ResourceAdaptationProcessor with
references to its interface. This would make it possible to have
alternative implementations or inject fake/mock implementations for
testing.
The VideoStreamAdapter is still responsible for constructing the
ResourceAdaptationProcessor, but beyond construction it is agnostic
towards the implementation.
With this CL, I claim https://crbug.com/webrtc/11222 complete.
TBR=ilnik@webrtc.org
Bug: webrtc:11222
Change-Id: I6e7a73bf1d0b5e97bc694f66180a747b27ffb018
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/174160
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Evan Shrubsole <eshr@google.com>
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#31148}
This CL adds miscellaneous unit tests for the
ResourceAdaptationProcessor, the VideoSourceRestrictions comparators and
the VideoStreamInputStateProvider.
Bug: webrtc:11172
Change-Id: If95f69644aaf2b43e3b19d5729bedef0b438c77b
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/174101
Reviewed-by: Evan Shrubsole <eshr@google.com>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#31147}
This CL is part of the Call-Level Adaptation Processing design doc:
https://docs.google.com/document/d/1ZyC26yOCknrrcYa839ZWLxD6o6Gig5A3lVTh4E41074/edit?usp=sharing
The ResourceListenerResponse was used to make the QualityScaler
not clear QP samples and instead increase its frequency of checking for
QP under certain circumstances, see enum description:
https://webrtc.googlesource.com/src.git/+/c70b1028d47c1aee4892545190cd66e97d09cd55/call/adaptation/resource.h#33
Because the QualityScaler depends on whether and how adaptation
happened it should listen to adaptation happening.
This CL moves the logic that was previously in VideoStreamAdapter closer
to the QualityScaler: QualityScalerResource::OnAdaptationApplied().
This would allow the VideoStreamAdapter to operate on a separate task
queue in the future, with no dependencies on any stream-specific
resources that might operate on other task queues.
Bug: webrtc:11172, webrtc:11521
Change-Id: I07971a8a5fab5715f4ccb7d2c63f1b92bd47170f
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/173090
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Evan Shrubsole <eshr@google.com>
Cr-Commit-Position: refs/heads/master@{#31143}
This CL breaks up the CheckQp() operation into several steps managed
by the inner helper class CheckQpTask, making responding to high or
low QP an asynchronous operation. Why? Reconfiguring the stream in
response to QP overuse will in the future be handled on a separate
task queue. See Call-Level Adaptation Processing for more details:
https://docs.google.com/document/d/1ZyC26yOCknrrcYa839ZWLxD6o6Gig5A3lVTh4E41074/edit?usp=sharing
Instead of "bool AdaptDown()" when high QP is reported,
synchronously returning true or false depending on the result of
adaptation, this CL introduces
void QualityScalerQpUsageHandlerInterface::OnReportQpUsageHigh(
rtc::scoped_refptr<QualityScalerQpUsageHandlerCallback>);
Where
QualityScalerQpUsageHandlerCallback::OnQpUsageHandled(
bool clear_qp_samples);
Instructs the QualityScaler whether to clear samples before
checking QP the next time or to increase the frequency of checking
(corresponding to AdaptDown's return value prior to this CL).
QualityScaler no longer using AdaptationObserverInterface, this class
is renamed and moved to overuse_frame_detector.h.
The dependency between CheckQpTasks is made explicit with
CheckQpTask::Result and variables like observed_enough_frames_,
adapt_called_ and adapt_failed_ are moved there and given more
descriptive names.
Bug: webrtc:11521
Change-Id: I7faf795aeee5ded18ce75eb1617f88226e337228
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/173760
Reviewed-by: Evan Shrubsole <eshr@google.com>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#31140}
We'll need to deprecate the previous classes due to being used externally
as an API.
Bug: webrtc:11489
Change-Id: I64de29c8adae304d0b7628e24dd0abc5be6387ba
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/173960
Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org>
Commit-Queue: Tommi <tommi@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#31136}
This CL is part of the Call-Level Adaptation Processing design doc:
https://docs.google.com/document/d/1ZyC26yOCknrrcYa839ZWLxD6o6Gig5A3lVTh4E41074/edit?usp=sharing
The ResourceUsageState was written as: {kOveruse, kStable, kUnderuse}.
The assumption was that if a resource neither wanted to adapt up or
down it would report kStable. But with the addition of
Resource::IsAdaptationUpAllowed() (prior CL) the notion of being
"stable" was already captured outside of ResourceUsageState.
Furthermore, kStable failed to capture what IsAdaptationUpAllowed() did
not: whether we can go up depends on the resulting resolution or frame
rate (restrictions_after). Perhaps we can go up a little, but not a lot.
This CL also adds Resource::ClearUsageState(). After applying an
adaptation, all usage states become invalidated (new measurements are
needed to know if we are still over- or underusing). This was always
the case, but prior to this CL this was not accurately reflected in the
Resource::usage_state() in-between measurements.
Bug: webrtc:11172
Change-Id: I140ff3114025b7732e530564690783e168d2509b
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/173088
Reviewed-by: Evan Shrubsole <eshr@google.com>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#31110}
This CL is part of the Call-Level Adaptation Processing design doc:
https://docs.google.com/document/d/1ZyC26yOCknrrcYa839ZWLxD6o6Gig5A3lVTh4E41074/edit?usp=sharing
This gets to the heart of unblocking call-level adaptation, largely
made possible due to the previous CLs in the chain.
The parts of the code that are responsible for responding to resource
usage signals, obtaining adaptations and applying them are moved to
ResourceAdaptationProcessor in call/adaptation/.
The parts of the code that are responsible for managing
VideoStreamEncoder-specific resources stay inside the
VideoStreamEncoderResourceManager class in video/adaptation/.
After this CL lands it should soon be possible to move the Processor
over to a separate task queue and let the Manager stay on the encoder
queue if PostTasks are added for communication between the two objects.
Bug: webrtc:11172
Change-Id: Ifa212467b4afd16e7ebfb9adfe17d2dca1cb7d67
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/173021
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Evan Shrubsole <eshr@google.com>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#31105}
This CL is part of the Call-Level Adaptation Processing design doc:
https://docs.google.com/document/d/1ZyC26yOCknrrcYa839ZWLxD6o6Gig5A3lVTh4E41074/edit?usp=sharing
By pushing VideoAdaptationCounters updates on VideoSourceRestrictions
changes, alongside the Resource* that triggered the adaptation, we are
able to update |active_counts_| without an explicit dependency on the
VideoStreamAdapter. This allows a future CL to split up "processor"
logic from "video stream encoder resource and active counts" logic,
which will ultimately be necessary in order to do processing on a
"processing queue" and encoder and stats logic on the "encoder queue".
If the restrictions got cleared by an API call
(ResetVideoSourceRestrictions() or SetDegradationPreference()) we pass
null as the "reason_resource". This allows is to clear the
active_counts_, and the code that invokes
OnVideoSourceRestrictionsUpdated() does not have to be aware of
active_counts_ (needed to split the processor module in two).
Bug: webrtc:11172
Change-Id: Icab6d5121c0ebd27d2a00f1bffc8191f8f05f562
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/173000
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Evan Shrubsole <eshr@google.com>
Cr-Commit-Position: refs/heads/master@{#31103}
This CL is part of the Call-Level Adaptation Processing design doc:
https://docs.google.com/document/d/1ZyC26yOCknrrcYa839ZWLxD6o6Gig5A3lVTh4E41074/edit?usp=sharing
The VideoStreamAdapter is currently responsible for aborting and not
providing adaptations if we are bitrate constrained
(kIsBitrateConstrained). Whether or not we are bitrate constrained is
clearly a resource question and should be phrased as such. By moving
this logic to Resource::IsAdaptationUpAllowed(), the VideoStreamAdapter
can continue to be thread-agnostic when a future CL introduces a
"processing queue", and the VideoStreamAdapter can be simplified: it
returns Adaptations even if we are constrained (but we refuse to Apply
them any resource rejects it).
This CL adds new Resource classes as inner classes of
ResourceAdaptationProcessor that take on the responsibility of
kIsBitrateConstrained logic:
PreventIncreaseResolutionDueToBitrateResource and
PreventAdaptUpInBalancedResource.
A third class, PreventAdaptUpDueToActiveCounts, also allows us to move
adaptation-aborting logic. This piece of code appears to be about not
adapting up if we’re already at the highest setting, which would be
VideoStreamAdapter responsibility (covered by
Adaptation::Status::kLimitReached), but it is actually more complicated
than that: the active_counts_ care about "reason", so it is really about
"is this resource type OK with you adapting up?". We should probably
rewrite this code in the future, but for now it is moved to an inner
class of ResourceAdaptationProcessor.
Other misc changes:
- ApplyDegradationPreference is moved to video_stream_adapter.[h/cc]
and renamed "Filter".
- OnResourceOveruse/Underuse now use Resource* as the reason instead of
AdaptReason. In a future CL, the processor will be split into a
"processor" part and a "video stream encoder resource manager" part.
Only the manager needs to know about AdaptReason since this is only
used for |active_counts_| and we want to get rid of it as much as
possible as it is not future-proof.
Bug: webrtc:11172
Change-Id: I2eba9ec3d717f7024c451aeb14635fe759551318
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/172930
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Evan Shrubsole <eshr@google.com>
Cr-Commit-Position: refs/heads/master@{#31099}
This CL is part of the Call-Level Adaptation Processing design doc:
https://docs.google.com/document/d/1ZyC26yOCknrrcYa839ZWLxD6o6Gig5A3lVTh4E41074/edit?usp=sharing
The "input state" of a VideoStream, needed for adaptation and
decision-making, are: source resolution and frame rate, codec type and
min pixels per frame (based on encoder scaling settings). These values
are modified on the encoder queue of the VideoStreamEncoder.
But in order to unblock call-level adaptation processing, where
adaptation and decision making happens off the encoder queue, a snapshot
of the input states need to be available at point of processing:
introducing the VideoStreamInputState.
In this CL, the VideoStreamInputStateProvider is added to provide input
state snapshots across threads based on input from VideoStreamEncoder
and VideoStreamEncoderObserver.
The input state's HasInputFrameSizeAndFramesPerSecond() can now be
DCHECKed inside the VideoStreamAdapter in favor of having less
Adaptation::Status codes. Whether input is "sufficient" for adaptation
is now the responsibility of the Processor. (Goal: adapter is purely a
Adaptation generator and apply-er.)
Somewhat tangental, this CL also deletes VideoStreamEncoder-specific
methods from ResourceAdaptationProcessorInterface making them an
implementation detail of ResourceAdaptationProcessor. In a future CL,
the "processor" will be split up into a "processor" part and a "video
stream encoder resource manager" part - more on that later.
Bug: webrtc:11172
Change-Id: Id9b158f569db0140b75360aaf0f7e2e28fb924f4
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/172928
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Evan Shrubsole <eshr@google.com>
Cr-Commit-Position: refs/heads/master@{#31098}
This CL is part of the Call-Level Adaptation Processing design doc:
https://docs.google.com/document/d/1ZyC26yOCknrrcYa839ZWLxD6o6Gig5A3lVTh4E41074/edit?usp=sharing
The VideoStreamAdapter is responsible for generating adaptation
suggestions according to its DegradationPreference. Today there is one
DegradationPreference that you set, but internally the preference it
uses to make decisions is EffectiveDegradationPreference() which
reinterprets “balanced” as “maintain-resolution” if screenshare is used.
By moving the “effective” logic to the ResourceAdaptationProcessor, the
VideoStreamAdapter will not need to know about the type of track, and
the responsibility of the adapter is minimized. The “effective” logic
is non-standard and something we want to get rid of - until then, it
should be the responsibility of the processor to configure the adapter
to use the appropriate strategy, rather than for the adapter to know
about more states of the system than it needs to.
Future CLs will further minimize what the adapter needs to know, moving
"decision-making" logic to the Processor and "is adapt up allowed?"
logic to the Resources.
By removing the VideoInputMode enum the VideoStreamAdapter does not
know if we have input which has to be checked externally. Input
handling is followed-up on in the next CL.
Bug: webrtc:11172
Change-Id: I37ec9e7392f835cf8fef9829a2c945183f0e9b65
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/172927
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Evan Shrubsole <eshr@google.com>
Cr-Commit-Position: refs/heads/master@{#31096}
Replaces this with 2 methods instead, adding clarity.
ClearAdaptationStats
- Resets the adaptations statistics to 0. This is done,
when the degredation is reset, for example when the preference
is changed to/from BALANCED.
UpdateAdaptationMaskingSettings
- Updates the settings for adaptation statistics reporting.
This way we don't report quality adaptations if quality scaling
is not enabled (same for resolution/fps scaling).
The adaptation counting inside the SendStatisticsProxy is
now done in a struct that counts the totals, and then masks
out these counts based on the adaptation settings. The
MaskedAdaptationSteps uses optionals to hide the values we
shoudn't report, while the AdaptationSteps always hold the real
totals.
All tests have been updated to use the Reset/Clear method as needed.
Now that AdaptationCounters and AdaptSteps use the same structure,
AdaptationCounters was moved to api/video and replaces AdaptSteps.
The AdaptReason enum is also redundant now, and will be removed
in a follow-up CL.
R=hbos@webrtc.org
Bug: webrtc:11392
Change-Id: Iaed6488581325d341a056b5bbf76a01c19d6c282
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/171685
Reviewed-by: Niels Moller <nisse@webrtc.org>
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Evan Shrubsole <eshr@google.com>
Cr-Commit-Position: refs/heads/master@{#31083}
https://webrtc-review.googlesource.com/c/src/+/172847
------------ original description --------------
Preparation for ReceiveStatisticsProxy lock reduction.
Update tests to call VideoReceiveStream::GetStats() in the same or at
least similar way it gets called in production (construction thread,
same TQ/thread).
Mapped out threads and context for ReceiveStatisticsProxy,
VideoQualityObserver and VideoReceiveStream. Added
follow-up TODOs for webrtc:11489.
One functional change in ReceiveStatisticsProxy is that when sender
side RtcpPacketTypesCounterUpdated calls are made, the counter is
updated asynchronously since the sender calls the method on a different
thread than the receiver.
Make CallClient::SendTask public to allow tests to run tasks in the
right context. CallClient already does this internally for GetStats.
Remove 10 sec sleep in StopSendingKeyframeRequestsForInactiveStream.
Bug: webrtc:11489
Change-Id: I491e13344b9fa714de0741dd927d907de7e39e83
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/173583
Commit-Queue: Tommi <tommi@webrtc.org>
Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#31077}
Save the frame transformer set on unsignaled receivers, and set the
transformer when the ssrc becomes known.
Pass the receiver's ssrc on registering the transformed frame callback,
to associate separate frame transformer sinks for each receiver.
Bug: chromium:1065838
Bug: chromium:1065838
Change-Id: I2a214bdb6cb9a8012928a03f046f311c344370f8
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/173201
Commit-Queue: Marina Ciocea <marinaciocea@webrtc.org>
Reviewed-by: Tommi <tommi@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#31051}
The RtpVideoStreamReceiverFrameTransformerDelegate::IsKeyFrame()
implementation was relying on the EncodedFrame::is_keyframe() API, which
checks the number of references to a frame. However the number of
references is not updated until after the frame is managed by the
receiver, after the frame has been transformed.
Update the delegate's IsKeyFrame() implementation to use the type of the
frame instead. The frame type is updated before transforming the frame,
on parsing the generic descriptor.
Bug: chromium:1068468
Change-Id: I84dadaecb1cd485262c2f1681dfa653d84693f69
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/173025
Commit-Queue: Marina Ciocea <marinaciocea@webrtc.org>
Reviewed-by: Magnus Flodman <mflodman@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#31024}
This reverts commit 24eed2735b2135227bcfefbabf34a89f9a5fec99.
Reason for revert: Speculative revert: breaks downstream project
Original change's description:
> Preparation for ReceiveStatisticsProxy lock reduction.
>
> Update tests to call VideoReceiveStream::GetStats() in the same or at
> least similar way it gets called in production (construction thread,
> same TQ/thread).
>
> Mapped out threads and context for ReceiveStatisticsProxy,
> VideoQualityObserver and VideoReceiveStream. Added
> follow-up TODOs for webrtc:11489.
>
> One functional change in ReceiveStatisticsProxy is that when sender
> side RtcpPacketTypesCounterUpdated calls are made, the counter is
> updated asynchronously since the sender calls the method on a different
> thread than the receiver.
>
> Make CallClient::SendTask public to allow tests to run tasks in the
> right context. CallClient already does this internally for GetStats.
>
> Remove 10 sec sleep in StopSendingKeyframeRequestsForInactiveStream.
>
> Bug: webrtc:11489
> Change-Id: Ib45bfc59d8472e9c5ea556e6ecf38298b8f14921
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/172847
> Commit-Queue: Tommi <tommi@webrtc.org>
> Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org>
> Reviewed-by: Magnus Flodman <mflodman@webrtc.org>
> Cr-Commit-Position: refs/heads/master@{#31008}
TBR=mbonadei@webrtc.org,henrika@webrtc.org,kwiberg@webrtc.org,tommi@webrtc.org,juberti@webrtc.org,mflodman@webrtc.org
# Not skipping CQ checks because original CL landed > 1 day ago.
Bug: webrtc:11489
Change-Id: I48b8359cdb791bf22b1a2c2c43d46263b01e0d65
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/173082
Reviewed-by: Artem Titov <titovartem@webrtc.org>
Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#31023}
Update tests to call VideoReceiveStream::GetStats() in the same or at
least similar way it gets called in production (construction thread,
same TQ/thread).
Mapped out threads and context for ReceiveStatisticsProxy,
VideoQualityObserver and VideoReceiveStream. Added
follow-up TODOs for webrtc:11489.
One functional change in ReceiveStatisticsProxy is that when sender
side RtcpPacketTypesCounterUpdated calls are made, the counter is
updated asynchronously since the sender calls the method on a different
thread than the receiver.
Make CallClient::SendTask public to allow tests to run tasks in the
right context. CallClient already does this internally for GetStats.
Remove 10 sec sleep in StopSendingKeyframeRequestsForInactiveStream.
Bug: webrtc:11489
Change-Id: Ib45bfc59d8472e9c5ea556e6ecf38298b8f14921
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/172847
Commit-Queue: Tommi <tommi@webrtc.org>
Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org>
Reviewed-by: Magnus Flodman <mflodman@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#31008}
Add a new frame interface to be used by frame transformers in Insertable
Streams. TransformableFrameInterface will replace
video_coding::EncodedFrame in a follow up CL, once downstream
dependecies are updated to use the new interface.
Until the functions using video_coding::EncodedFrame are removed from
the API, the video sender and receiver frame transformer delegates call
both function versions to avoid breaking tests downstream.
The TransformableFrameInterface will be used for both audio and video
frame transformers in follow-up CLs.
Bug: webrtc:11380
Change-Id: I9389a8549c156e13b1d8c938ff51eaa69c502f33
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/171863
Commit-Queue: Marina Ciocea <marinaciocea@webrtc.org>
Reviewed-by: Magnus Flodman <mflodman@webrtc.org>
Reviewed-by: Karl Wiberg <kwiberg@webrtc.org>
Reviewed-by: Danil Chapovalov <danilchap@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30941}
This CL fixes a few issues where the reported fraction of frames
allocated to various temporal layers could be incorrect:
* In LibvpxVp8Encoder, calling GetEncoderInfo() while not initialized,
or when first configuring with temporal layers and then without,
could trigger incorrect fps allocations.
* In VP9 when different spatial layers have different max framerates,
the layer fps should be compared to the layer with the highest
configured fps, not codec_.maxFramerate which is updated to the
current input fps on SetRates().
* In EncoderBitrateAdjuster, just warn and ignore if a layer has
non-zero bps but zero fps, rather than passing down the chain and
risk weird behavior or divide by zero.
Bug: b/152040235
Change-Id: I548fb3e099b1ec9f536a7b93313fb40c4d32e596
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/171516
Commit-Queue: Erik Språng <sprang@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30880}
--- Background ---
The webrtc::VideoSendStream::StreamStats are converted into
VideoSenderInfo objects which turn into "outbound-rtp" stats objects in
getStats() (or "ssrc" objects in legacy getStats()).
StreamStats are created for each type of substream: RTP media streams,
RTX streams and FlexFEC streams - each with individual packet counters.
The RTX stream is responsible for retransmissions of a referenced media
stream and the FlexFEC stream is responsible for FEC of a referenced
media stream. RTX/FEC streams do not show up as separate objects in
getStats(). Only the media streams become "outbound-rtp" objects, but
their packet and byte counters have to include the RTX and FEC counters.
--- Overview of this CL ---
This CL adds MergeInfoAboutOutboundRtpSubstreams(). It takes
StreamStats of all kinds as input, and outputs media-only StreamStats
- incorporating the RTX and FEC counters into the relevant media
StreamStats.
The merged StreamStats objects is a smaller set of objects than the
non-merged counterparts, but when aggregating all packet counters
together we end up with exact same packet and count as before.
Because WebRtcVideoSendStream::GetVideoSenderInfo() currently aggregates
the StreamStats into a single VideoSenderInfo (single "outbound-rtp"),
this CL should not have any observable side-effects. Prior to this CL:
aggregate StreamStats. After this CL: merge StreamStats and then
aggregate them.
However, when simulcast stats are implemented (WIP CL:
https://webrtc-review.googlesource.com/c/src/+/168120) each RTP media
stream should turn into an individual "outbound-rtp" object. We will
then no longer aggregate all StreamStats into a single "info". This CL
unblocks simulcast stats by providing StreamStats objects that could be
turned into individual VideoSenderInfos.
--- The Changes ---
1. Methods added to RtpConfig to be able to easily tell the relationship
between RTP, RTX and FEC ssrcs.
2. StreamStats gets a StreamType (kMedia, kRtx or kFlexfec) that
replaces the booleans (is_rtx, is_flexfec).
3. "referenced_media_ssrc" is added to StreamStats, making it possible
to tell which kRtx/kFlexFec stream stats need to be merged with which
kMedia StreamStats.
4. MergeInfoAboutOutboundRtpSubstreams() added and used.
Bug: webrtc:11439
Change-Id: Iaf9002041169a054ddfd32c7ea06bd1dc36c6bca
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/170826
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Harald Alvestrand <hta@webrtc.org>
Reviewed-by: Stefan Holmer <stefan@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30869}
In preparation for a change that rely on payload type beeing present.
As side effect, fix test related to RED payload type.
Bug: None
Change-Id: I42f8460fbec578184da058c1f6b9620d497d576b
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/171508
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Commit-Queue: Danil Chapovalov <danilchap@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30864}
This patch extends the NetworkRoute struct with more information
about local/remote endpoints. It adds
- adapter type
- adapter id
- relay
(previously it was "only" network_id)
The patch leaves the {local/remote}_network_id fields
around and populated since downstream projects depend
on them. They will be removed once they have migrated.
OWNER: srte@ call/ test/
OWNER: asapersson@ video/
OWNER: hta@ p2p/ pc/ rtc_base/
BUG: webrtc:11434
Change-Id: I9bcec385b40d707db385fef40b2c7a315dd35dd0
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/170628
Reviewed-by: Karl Wiberg <kwiberg@webrtc.org>
Reviewed-by: Harald Alvestrand <hta@webrtc.org>
Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org>
Reviewed-by: Åsa Persson <asapersson@webrtc.org>
Commit-Queue: Jonas Oreland <jonaso@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30848}
this is a step towards resolving own todo: making AssembleFrame part of
the VideoRtpDepacketizer interface and replacing codec check with a
call to a virtual function.
RtpVideoStreamReceiver has access to the VideoRtpDepacketizers,
PacketBuffer - hasn't.
Bug: None
Change-Id: I83df09975c092bdb71bab270ced356d79a50683d
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/168056
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Philip Eliasson <philipel@webrtc.org>
Commit-Queue: Danil Chapovalov <danilchap@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30833}
The MockFecController is spitting out lots of warnings, as it is
being called when we don't care about it, in normal tests. Making
it a NiceMock allows it to receive calls without expectation and
not warn.
Bug: None
Change-Id: I1ea219c4665d86917718692dc013ae3ac47222ad
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/170600
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Elad Alon <eladalon@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Evan Shrubsole <eshr@google.com>
Cr-Commit-Position: refs/heads/master@{#30820}
This code was previously only exercised by
video_stream_adapter_unittest.cc and other tests, acting more like
integration tests than unit tests. Now that the VideoStreamAdapter is
in a good state, more extensive test coverage is added.
Testing includes:
- Default restrictions.
- Adapting up or down in "maintain-framerate", "maintain-resolution"
and "balanced", including...
- expecting how frame rate and/or resolution is affected,
- reaching kLimitReached,
- and reaching unrestricted.
- That "disabled" does not adapt.
- When adaptation is not possible, including...
- kInsufficientInput
- kAwaitingPreviousAdaptation
- kIsBitrateConstrained
- PeekNextRestrictions()
- "balanced" + "screenshare" = "maintain-resolution"
- Change degradation preference to/from "balanced" clears restrictions.
- That using invalidated adaptations triggers DCHECKs.
Bug: webrtc:11393
Change-Id: I28e2cf227bc1fd8871ee0d18d9570d4063449160
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/170625
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Evan Shrubsole <eshr@google.com>
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30816}
It was possible before if an input fps dropped due to cpu adaptation
Also, this CL removes occasional test failure (it could've happened if
input framerate got very low)
Bug: webrtc:11432,webrtc:11426
Change-Id: Id1a4df23302f7b8ab6781f1e7cca5112bfcfe9ea
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/170469
Reviewed-by: Artem Titov <titovartem@webrtc.org>
Commit-Queue: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30802}
This CL introduces the Adaptation class used by VideoStreamRestrictor.
This refactors the AdaptationTarget, AdaptationTargetOrReason,
CannotAdaptReason and AdaptationAction.
What is publicly exposed is simply a Status code. If it's kValid then
we can adapt, otherwise the status code describes why we can't adapt
(just like CannotAdaptReason prior to this CL). This means
AdaptationTargetOrReason is no longer needed. Target+reason are merged.
The other classes are renamed and moved and put in the private
namespace of Adaptation: Only the VideoStreamAdapter (now a friend
class of Adaptation) and its inner class VideoSourceRestrictor needs to
know how to execute the adaptation.
Publicly, you can now tell the effects of the adaptation without
applying it with PeekNextRestrictions() - both current and next steps
are described in terms of VideoSourceRestrictions. The rest are hidden.
This would make it possible, in the future, for a Resource to accept or
reject a proposed Adaptation by examining the resulting frame rate and
resolution described by the resulting restrictions. E.g. even if we are
not overusing bandwidth at the moment, the BW resource can prevent us
from applying a restriction that would exceed the BW limit before we
apply it.
This CL also moves input to a SetInput() method, and Increase/Decrease
methods of VideoSourceRestrictor are made private in favor of
ApplyAdaptationSteps().
Bug: webrtc:11393
Change-Id: Ie5e2181836ab3713b8021c1a152694ca745aeb0d
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/170111
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Evan Shrubsole <eshr@google.com>
Cr-Commit-Position: refs/heads/master@{#30794}
These are considered expandable, and since video tests are very
expensive (45s each), let's remove them.
Bug: webrtc:11426
Change-Id: I4aea18e93d3b3672900650aacf0b5524c52c2900
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/170364
Commit-Queue: Patrik Höglund <phoglund@webrtc.org>
Commit-Queue: Patrik Höglund <phoglund@google.com>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30790}
This introduces AdaptationTargetOrReason and gets rid of
VideoStreamAdapter's dependency on the VideoStreamEncoderObserver.
AdaptationTargetOrReason provides information about why an adaptation
target could not be returned from GetAdaptUpTarget() and
GetAdaptDownTarget() with the enum CannotAdaptReason and the boolean
min_pixel_limit_reached.
While the enum value is not used by the caller in this CL, it makes
explicit reasons the adapter is allowed to reject a target. TODOs are
added documenting how we want to get rid of kAwaitingPreviousAdaptation
for multi-stream use cases and how kIsBitrateConstrained can be
rephrased as a resource problem in the future.
min_pixel_limit_reached() allows us to move the responsibility of stats
reporting to the module. A TODO documents how this could be replaced by
kLimitReached or similar logic in the future.
Bug: webrtc:11393
Change-Id: Iffdd8ddb01641937741fac353174ea14168477ce
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/169928
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Evan Shrubsole <eshr@google.com>
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30789}
