Commit Graph

1588 Commits

Author SHA1 Message Date
f12231d742 Add wildcard visibility to video_replay to make it buildable in Chromium.
Bug: chromium:942546
Change-Id: Ib798b58e854a2471ab1bb94725cb0ee2b04b84da
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/168320
Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org>
Commit-Queue: Max Moroz <mmoroz@chromium.org>
Cr-Commit-Position: refs/heads/master@{#30477}
2020-02-06 21:41:31 +00:00
065348503c [Overuse] Move EncodeUsageResource/QualityScalerResource to own files.
This CL changes EncodeUsageResource and QualityScalerResource from
private inner classes of OveruseFrameDetectorResourceAdaptationModule to
standalone classes, moving them into separate files.

This CL does not intend to change any lines of code, only move them.
Except for removing an unused method quality_scaler().

Bug: webrtc:11222
Change-Id: I86bf7eb78c80031888c403ac43c2bdf9b24eaea6
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/168198
Reviewed-by: Evan Shrubsole <eshr@google.com>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30472}
2020-02-06 14:08:39 +00:00
48258acabf [Overuse] Implement Resource and ResourceUsageListener.
The Resource interface (previously a skeleton not used outside of
testing) is updated to inform listeners of changes to resource
usage. Debugging methods are removed (Name, UsageUnitsOfMeasurements,
CurrentUsage). The interface is implemented by
OveruseFrameDetectorResourceAdaptationModule's inner classes
EncodeUsageResource and QualityScalerResource.

The new ResourceUsageListener interface is implemented by
OveruseFrameDetectorResourceAdaptationModule. In order to avoid adding
AdaptationObserverInterface::AdaptReason to the ResourceUsageListener
interface, the module figures out if the reason is "kCpu" or "kQuality"
by looking which Resource object triggered
OnResourceUsageStateMeasured(). These resources no longer need an
explicit reference to OveruseFrameDetectorResourceAdaptationModule and
could potentially be used by a different module.

In this CL, AdaptationObserverInterface::AdaptDown()'s return value is
still needed by QualityScaler. This is mirrored in the return value of
ResourceUsageListener::OnResourceUsageStateMeasured(). A TODO is added
to remove it and a comment explains how the current implementation
seems to break the contract of the method (as was the case prior to
this CL).

Follow-up work include:
- Move EncodeUsageResource and QualityScalerResource to separate files.
- Make resources injectable, allowing fake resources in testing and
  removing OnResourceOveruseForTesting() methods.
  (Investigate adding the necessary input signals to the Resource
  interface or relevant sub-interfaces so that the module does not need
  to know which Resource implementation is used.)
- And more! See whiteboard :)

Bug: webrtc:11222
Change-Id: I0a46ace4a2e617874e3ee97e67e3a199fef420a2
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/168180
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Reviewed-by: Evan Shrubsole <eshr@google.com>
Cr-Commit-Position: refs/heads/master@{#30469}
2020-02-06 12:45:14 +00:00
7875c99e82 [Overuse] Add EncodeUsageResource and QualityScalerResource.
This refactors the usage of OveruseFrameDetector in
OveruseFrameDetectorResourceAdaptationModule into an inner class of the
module, making the interaction between the detector and the module the
responsibility of this helper class instead.

Similarly, QualityScaler usage is moved into QualityScalerResource.

This takes us one step closer to separate the act of detecting
overuse/underuse of a resource and the logic of what to do when
overuse/underuse happens.

Follow-up CLs should build on this in order to materialize the concept
of having resources, streams and a central decision-maker deciding how
to reconfigure the streams based on resource usage state.

Bug: webrtc:11222
Change-Id: I99a08a42218a871db8f477f31447a6379433ad05
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/168057
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Evan Shrubsole <eshr@google.com>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30468}
2020-02-06 11:29:02 +00:00
a9e1026304 Make video_replay buildable from Chromium.
Bug: chromium:942546
Change-Id: Ic127e74b75ccb1fa65b317711d20344d0caee5fb
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/168280
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Commit-Queue: Mirko Bonadei <mbonadei@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30467}
2020-02-06 10:55:22 +00:00
e331a122aa Move quality rampup experiment to overuse module
Bug: webrtc:11222
Change-Id: I8d0860bfe8bdfe0a051f5a6165cdcfa0cc25cfb5
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/168181
Reviewed-by: Erik Språng <sprang@webrtc.org>
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Commit-Queue: Evan Shrubsole <eshr@google.com>
Cr-Commit-Position: refs/heads/master@{#30465}
2020-02-06 08:38:39 +00:00
7c3a1fc082 Move initial quality experiment to adaptation module
Bug: webrtc:11222
Change-Id: Iaa33bd6369a11f91e677b015eb2db412d0fbff23
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/168053
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Evan Shrubsole <eshr@google.com>
Cr-Commit-Position: refs/heads/master@{#30456}
2020-02-05 10:10:22 +00:00
c81798b0c4 Configure QP scaler in adaptation module
Bug: webrtc:11222
Change-Id: Ia50ba3d024d0cbbaeddf8bf67ee652be602c5df9
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/168052
Reviewed-by: Erik Språng <sprang@webrtc.org>
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Commit-Queue: Evan Shrubsole <eshr@google.com>
Cr-Commit-Position: refs/heads/master@{#30453}
2020-02-04 14:46:06 +00:00
f2be3eff26 Move initial frame drop to overuse module
It would be nice for this to stay in video stream encoder,
but this feature is mostly related to quality scaling. Perhaps
something easier to understand is possible in the future.

Bug: webrtc:11222
Change-Id: I71705f33ff94bbcf2fb9b5c94226c8e76dcba94c
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/168051
Commit-Queue: Evan Shrubsole <eshr@google.com>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30446}
2020-02-03 11:56:31 +00:00
c809e8bd62 Move quality scaling frame drop logic to adaptation module
Bug: webrtc:11222
Change-Id: I43db57fa128924ccaa3e44cd58098e7938e5ff5e
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/168050
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Evan Shrubsole <eshr@google.com>
Cr-Commit-Position: refs/heads/master@{#30445}
2020-02-03 10:55:11 +00:00
bfe3ef8feb Report frame qp to quality scaler via overuse module
Bug: webrtc:11222
Change-Id: I63938adf5f623429eab1bcd668cde8fa5a1a083a
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/167924
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30435}
2020-01-31 09:18:28 +00:00
cf0595234c Move quality scaler into adaptation module
This allows for further refactoring, eventually moving
all of quality scaler out of video stream encoder.

Bug: webrtc:11222
Change-Id: Id121608da56f57549a616ccc5f141bb598668b40
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/167728
Commit-Queue: Evan Shrubsole <eshr@google.com>
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30417}
2020-01-30 09:05:19 +00:00
73a5e916a9 Remove task_queue dependency for QualityScaler
This allows for the possiblity to move the QualityScaler
out of the VideoStreamEncoder in the future.


Bug: webrtc:11222
Change-Id: I1d563cf08791e27ff5065ce90bcb150a7974d868
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/167534
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Evan Shrubsole <eshr@google.com>
Cr-Commit-Position: refs/heads/master@{#30406}
2020-01-29 12:14:10 +00:00
97ffbefdab Pass and store PacketBuffer::Packet by unique_ptr
to avoid expensive move of the Packet and prepare PacketBuffer
to return list of packets as a frame.

Bug: None
Change-Id: I19f0452c52238228bbe28284ebb197491eb2bf4e
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/167063
Reviewed-by: Philip Eliasson <philipel@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Danil Chapovalov <danilchap@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30404}
2020-01-29 11:48:55 +00:00
ad515a255b [Overuse] Move GetCpuOveruseOptions() to adaption module.
This removes the last remaining explicit reference from
OveruseFrameDetectorResourceAdaptationModule to
VideoStreamEncoder.

VideoStreamEncoder's call to SetEncoderSettings() inside
ReconfigureEncoder() is moved a few lines down - it was discovered that
during these lines the EncoderInfo config could get modified in
response to InitEncode() - so this fixes a potential bug.

Bug: webrtc:11222
Change-Id: I9746f28a4df8e631e297669c10636bf17b39acec
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/167363
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Evan Shrubsole <eshr@google.com>
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30381}
2020-01-27 13:51:40 +00:00
159c414ff8 Detach LossNotificationController from RtpGenericFrameDescriptor
To allow to use the LossNotificationController with
an updated version of the frame descriptor extension

Bug: webrtc:10342
Change-Id: I5ac44dc5549dfcfc73bf81ad1e8eab8bd5dd136e
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/166166
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Philip Eliasson <philipel@webrtc.org>
Reviewed-by: Elad Alon <eladalon@webrtc.org>
Commit-Queue: Danil Chapovalov <danilchap@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30369}
2020-01-24 11:53:28 +00:00
cdd73e095c Migrate PC level tests on new video codec configuration API
Bug: webrtc:10138
Change-Id: I7129857724aafbfae71f36554ef79be78c442cb6
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/167066
Commit-Queue: Artem Titov <titovartem@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30362}
2020-01-23 13:45:33 +00:00
02d51f9fdc Remove unused field trial WebRTC-InitialFramedrop
Bug: webrtc:9176, webrtc:6086
Change-Id: Ie02800963f790f07b4c60ff01a04ecd6b5e1113d
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/167181
Reviewed-by: Kári Helgason <kthelgason@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Commit-Queue: Evan Shrubsole <eshr@google.com>
Cr-Commit-Position: refs/heads/master@{#30361}
2020-01-23 13:42:03 +00:00
d4578ae962 [Overuse] Encoding pipeline as input signals in the abstract interface.
This defines the following methods:
- OnFrame(), replaces SetLastFramePixelCount().
- OnFrameDroppedDueToSize(), a rename of FrameDroppedDueToSize() to
  match the other methods.
- OnEncodeStarted(), a rename of the incorrectly named FrameCaptured().
- OnEncodeCompleted(), a rename of the poorly named FrameSent().

In order to get rid of SetLastFramePixelCount(), the "we don't know the
frame size" use case - which was previously implicitly avoided by
invoking SetLastFramePixelCount() with a made-up value for
last_frame_info_ - is now avoided using ".value_or()" in
LastInputFrameSizeOrDefault(). This does mean that a constant 144p
resolution value is referenced in two places, but the fact that this is
a magic value is at least made explicit. This may help future
improvements.

Bug: webrtc:11222
Change-Id: I3b28daa8c5ecf57c6537957d4759f15e24bb2234
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/166961
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Evan Shrubsole <eshr@google.com>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30352}
2020-01-22 17:11:20 +00:00
2bc91e8c6a Avoid extra EncodedFrame copy in RunPostEncode
All uses of encoded_image are const, except for the copy for running on
the encoder_queue_.

Bug: None
Change-Id: I7fc8cb46f6afb42a2d27961d3d3ff8d9e63fe1b5
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/166442
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Evan Shrubsole <eshr@google.com>
Cr-Commit-Position: refs/heads/master@{#30351}
2020-01-22 15:41:38 +00:00
ede69c0fbe [Overuse] Setting the target bitrate through the interface.
The poorly named SetEncoderStartBitrate() is renamed
SetEncoderTargetBitrate() and added to the abstract resource adaptation
module interface.

The so-called "start bitrate" was updated to match the target bitrate,
so this was only ever a "start bitrate" until we had any estimates. The
variable is renamed in VideoStreamEncoder as well, and usage of optional
types are introduced to avoid magical values in a few places in the
existing code.

Bug: webrtc:11222
Change-Id: Idde92f68f34616aa3c34ab77a791fdbe7ea7af26
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/166880
Reviewed-by: Evan Shrubsole <eshr@google.com>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30347}
2020-01-22 13:38:38 +00:00
33aaa35d54 Fix video_replay to build and actually work
Add it to default build target, so it won't get broken accidentally
again. Fix configuration issue with field trials (new parameter was
added recently, but wasn't set by video_replay)

Bug: webrtc:11287
Change-Id: I9c18746d899acd7ac68c1b9b3a646b862c41897a
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/166900
Commit-Queue: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30345}
2020-01-22 13:16:28 +00:00
cd02ebaea0 Use intersection of app and encoder bitrate limits.
Before this change, if both app and encoder provided bitrate limits,
WebRTC ignored the limits provided by encoder. Now intersection of
these sets is used.

Also changed DCHECKs in GetEncoderBitrateLimits to allow zero values
of min_bitrate_bps and min_start_bitrate_bps.

Bug: none
Change-Id: Ib8be965ea43f51013b0a0f82fd4256a372432dda
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/166600
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Rasmus Brandt <brandtr@webrtc.org>
Commit-Queue: Sergey Silkin <ssilkin@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30338}
2020-01-21 15:16:29 +00:00
43bfe0b8a6 Enforce VideoEncoderConfig.num_temporal_layers >= 1.
This change clarifies the semantics of this field:
  unset: Depends on context.
  == 0: Invalid.
  == 1: No temporal layering.
  >= 2: Temporal layering.

We should try to remove the wrapping optional later.

Bug: webrtc:11297
Change-Id: Id765f2dc1d31a4ba3cd424978ac6054cd60152ea
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/166528
Commit-Queue: Rasmus Brandt <brandtr@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30336}
2020-01-21 13:38:08 +00:00
4bab2fcf6b [Overuse] Setting encoder configurations through the interface.
This squashes together several input signals that were spread out
through several calls into a single method and calling place:
SetEncoderSettings(), invoked from ReconfigureEncoder(). This is added
to the abstract interface.

This makes the following methods obsolete which are removed:
- SetEncoder(): The VideoEncoder was only used for GetEncoderInfo();
  the VideoEncoder::EncoderInfo is now part of the EncoderSettings.
- SetEncoderConfig(): The VideoEncoderConfig is part of
  EncoderSettings. The config is used for its codec_type and
  content_type enums.
- SetCodecMaxFrameRate(): The max frame rate was the same as
  VideoCodec::maxFramerate. VideoCodec is now part of EncoderSettings.

There may be some overlap in information between EncoderConfig and
VideoCodec, but that is outside the scope of this CL, which only makes
sure to bundle encoder settings-like information into one input signal.

Bug: webrtc:11222
Change-Id: I67c49c49c0a859cb7d5051939a461593c695a789
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/166602
Reviewed-by: Erik Språng <sprang@webrtc.org>
Reviewed-by: Evan Shrubsole <eshr@google.com>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30332}
2020-01-21 11:48:11 +00:00
6c13fd9071 Move bandwidth overuse detection out of VideoStreamEncoder
Bug: webrtc:11222
Change-Id: I12ccd008c848a0146fb22292f8dac46d1f7be9b4
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/166531
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Commit-Queue: Evan Shrubsole <eshr@google.com>
Cr-Commit-Position: refs/heads/master@{#30329}
2020-01-21 10:24:31 +00:00
8b1338bf8e Propagate is_bw_limited flag with bw allocation everywhere it's copied
Bug: webrtc:11015
Change-Id: Ie971d29d8a8d140ba120a51dd3920291034a4d48
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/166526
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30325}
2020-01-20 16:12:03 +00:00
fae6f0e87b [Overuse] MaybeUpdateTargetFrameRate() & ResetVideoSourceRestrictions()
This CL does two things for the sake of getting us closer to adaptation
modules being injectable and usable without knowing implementation
details.

Firstly, RefreshTargetFramerate() is removed. The target frame rate is
dependent on two things: 1) the codec max frame rate, and 2) the video
source restrictions. If either of these two changes, the target frame
rate is updated - there is no need to trigger this externally; the
module already knows if either of these factors change.
The private method MaybeUpdateTargetFrameRate() is added to ensure
overuse_detector->OnTargetFramerateUpdated() happens when necessary.

In doing this, the frame rates are updated to use
absl::optional<double>. This documents its optionality and avoids
magical values (previously -1 was not a bug but meaning "missing"). It
also matches VideoSourceRestrictions::max_frame_rate()'s type.

Secondly, ResetAdaptationCounters() is renamed
ResetVideoSourceRestrictions(). This more accurately describes what it
is doing; it is resetting the restrictions (the adaptation counters
getting reset is merely an implementation specific side-effect of
this). This method is added to the generic interface.

The usefulness of being able to ResetVideoSourceRestrictions() is
questioned in a TODO - current usage of this is when "quality rampup"
finishes. Nevertheless, any module could implement this functionality
so it belongs to the interface for now.

Bug: webrtc:11222
Change-Id: I079785df55fc9894e85087ec98be3e4ebd0713c5
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/166522
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Evan Shrubsole <eshr@google.com>
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30320}
2020-01-20 12:58:38 +00:00
a3d4252a01 ResourceAdaptationModule HasInputVideo+DegradationPreference setters.
The "has_input_video_ logic" is simplified to abort AdaptUp() and
AdaptDown() directly rather than in each calling place of the
VideoSourceRestrictor. The intent is no change in behavior.
The degradation_preference_ is removed from the VideoSourceRestrictor
as its only usage was DCHECKing (not worth it).

ResourceAdaptationModuleInterface gets SetHasInputVideo() and
SetDegradationPreference(), making these things controllable without
knowing implementation details.

StartCheckForOveruse() and StopCheckForOveruse() are renamed to
StartResourceAdaptation() and StopResourceAdaptation().

Bug: webrtc:11222
Change-Id: Id2d7f34d427dfb3ecd4831b1a245d07becae6520
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/166173
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Evan Shrubsole <eshr@google.com>
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30296}
2020-01-17 10:43:02 +00:00
64f1f3f04e Replace RTC_FALLTHROUGH with ABSL_FALLTHROUGH_INTENTED
Bug: None
Change-Id: I7287403f3fb13b8e30f92ca3cf1882b03bb53a6e
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/166176
Reviewed-by: Karl Wiberg <kwiberg@webrtc.org>
Commit-Queue: Danil Chapovalov <danilchap@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30283}
2020-01-16 15:20:35 +00:00
07b17df771 Move DegradationPreference logic to the encoder queue.
This moves SetHasInputVideoAndDegradationPreference() to the encoder
queue. OveruseFrameDetectorResourceAdaptationModule is now entirely
single-threaded, including its inner class VideoSourceRestrictor.

VideoStreamEncoder now protects the module with RTC_GUARDED_BY. This
ensures it is safely used, even without a SequenceChecker inside of the
module. The module's |encoder_queue_| is removed.

The one task queue reference that is needed - passing down the current
task queue to StartCheckForOveruse() - is replaced by a TaskQueueBase*
(instead of rtc::TaskQueue*), enabling obtaining the current queue with
TaskQueueBase::Current(). (There is no rtc::TaskQueue::Current().)

Furthermore, the only uses of VideoSourceSinkController that isn't on
the encoder queue are documented, with a TODO saying if these are moved
the VideoSourceSinkController could also be made single-threaded.
However since this requires introducing a delay to
VideoStreamEncoder::SetSource() and VideoStreamEncoder::Stop(),
arguably a more risky change, if this is to be attempted that should be
in a separate CL.

Bug: webrtc:11222
Change-Id: I448ca5125708d5f66b95b0b180d6d24cc356dfa9
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/165783
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Evan Shrubsole <eshr@google.com>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30263}
2020-01-15 11:58:04 +00:00
b2b2031457 Concatenate string literals at compile time.
This CL was generated by running:
git ls-files | grep ".cc" | xargs perl -i -ne 'BEGIN {undef $/}; s/("[\s\n]*<<[\s\n]*")/" "/g; print;'; git cl format

After that I manually edited modules/audio_processing/gain_controller2.cc to preserve its original
formatting.

This primary benefit of this change is a small reduction in binary size.

Bug: None
Change-Id: I689fa7ba9c717c314bb167e5d592c3c4e0871e29
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/165961
Reviewed-by: Alessio Bazzica <alessiob@webrtc.org>
Reviewed-by: Karl Wiberg <kwiberg@webrtc.org>
Commit-Queue: Jonas Olsson <jonasolsson@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30251}
2020-01-14 14:47:48 +00:00
8234b92ba3 Move DegradationPreference logic out of VideoSourceSinkController.
The DegradationPreference logic is moved into
OveruseFrameDetectorResourceAdaptationModule. This makes the adaptation
module solely responsible for degradation preference, and the
VideoStreamEncoder the only bridge between the adaptation module and the
VideoSourceSinkController.

The adaptation module is now unaware of the existence of a controller.
It only "speaks" VideoSourceRestrictions, which is a big milestone in
making adaptation modules injectable.

A follow-up CL will explore the possibility of reconfiguring the
controller's source and which degradation preference to use to the
encoder queue. This would allow us to make several classes
single-threaded, but it is a change in behavior and should be done in a
separate CL.

Bug: webrtc:11222
Change-Id: Ib7f640e12789da5f801177926c2072a51818f261
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/165684
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Evan Shrubsole <eshr@google.com>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30237}
2020-01-13 17:24:48 +00:00
ce0ea49001 VideoStreamEncoder configuring source/sink with VideoSourceController.
This is part of the work for making VideoStreamEncoder responsible for
configuring its source/sink and limiting the responsibility of
OveruseFrameDetectorResourceAdaptationModule to only output relevant
VideoSourceRestrictions.

BEFORE THIS CL

Prior to this CL, OveruseFrameDetector was responsible for performing
AddOrUpdateSink() on the source, which it did using its nested class
VideoSourceProxy.

AddOrUpdateSink() could happen for both adaptation and non-adaptation
related reasons. For example:
- Adaptation related: AdaptUp() or AdaptDown() happens, causing updated
  VideoSourceRestrictions.
- Non-adaptation related: VideoStreamEncoder asks the module to
  reconfigure the source/sink for it, such as with
  SetMaxFramerateAndAlignment() or SetWantsRotationApplied().

AFTER THIS CL

AddOrUpdateSink() is performed by VideoSourceController, which is owned
by VideoStreamEncoder. Any reconfiguration has to go through the
VideoStreamEncoder. This means that:
- Non-adaptation related settings happen between VideoStreamEncoder and
  VideoSourceController directly (without going through the adaptation
  module).
- Adaptation related changes can be expressed in terms of
  VideoSourceRestrictions. OveruseFrameDetectorResourceAdaptationModule
  only has to output the restrictions and not know or care about other
  source/sink settings.

For now, VideoSourceController has to know about DegradationPreference.
In a future CL, the DegradationPreference logic should move back to
the adaptation module. The VideoSourceRestrictions are fully capable of
expressing all possible source/sink values without the "modifier" that
is the degradation preference.

Bug: webrtc:11222
Change-Id: I0f058c4700ca108e2d9f212e38b61f6f728aa419
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/162802
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Evan Shrubsole <eshr@google.com>
Cr-Commit-Position: refs/heads/master@{#30228}
2020-01-13 11:14:04 +00:00
d238200882 Introduce ResourceAdaptationModuleListener and VideoSourceRestrictions.
The VideoSourceRestrictions describe the maximum pixels per frame and
max frame rate of a video source.

This CL makes the ResourceAdaptationModuleInterface responsible for the
reconfiguration of video sources. The VideoSourceRestrictions is the
output of an adaptation module, and the ResourceAdaptationModuleListener
handles the callback for when the source restrictions change.

The OveruseFrameDetectorResourceAdaptationModule is updated to output
its changes using these interfaces, and VideoStreamEncoder - now a
listener - is made responsible for triggering the reconfiguring the
video source.

Performing the reconfiguration still requires interacting with the
VideoSourceProxy - it is still partially responsible for keeping track
of rtc::VideoSinkWants settings and performing AddOrUpdateSink(). For
now this may look a bit weird: the VideoSourceProxy tells the
VideoStreamEncoder about the new restrictions, and then the
VideoStreamEncoder tells the VideoSourceProxy to apply these
restrictions on the source/sink. This exercises the listener though, and
unblocks the next CL.

The next CL should move all "configuring the source" logic to the
VideoStreamEncoder instead, so that the only information that is tracked
by OveruseFrameDetectorResourceAdaptationModule is what it actually
outputs to the listener. See the next CL
(https://webrtc-review.googlesource.com/c/src/+/162802) where a
VideoSourceController is introduced, to be owned by the
VideoStreamEncoder rather than the adaptation module.

Bug: webrtc:11222
Change-Id: I450ce74f51d96c4b98009a06134db671893d8fdc
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/162522
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Evan Shrubsole <eshr@google.com>
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30227}
2020-01-13 10:57:00 +00:00
539f9b376e Use a TaskQueue for decoding in VideoStreamDecoderImpl.
Long term goal is to use the VideoStreamDecoder in the VideoReceiveStream so
that we can stop using legacy VideoCodingModule components and classes. This CL is
one of several in preparation for that.

Bug: webrtc:7408, webrtc:9378
Change-Id: Ifd7e4c3c7d38dbb7c4b0636aaad318c571a29158
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/164525
Reviewed-by: Sebastian Jansson <srte@webrtc.org>
Reviewed-by: Åsa Persson <asapersson@webrtc.org>
Reviewed-by: Danil Chapovalov <danilchap@webrtc.org>
Commit-Queue: Philip Eliasson <philipel@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30211}
2020-01-10 14:31:22 +00:00
5c35f2fb1b Delete RtpDepacketizerVp9 in favor of VideoRtpDepacketizerVp9
Bug: webrtc:11152
Change-Id: Ic50f2dc49ca420b3406d4dea11ed20328aa59136
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/165382
Commit-Queue: Danil Chapovalov <danilchap@webrtc.org>
Reviewed-by: Sam Zackrisson <saza@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30195}
2020-01-09 13:07:44 +00:00
f5e71e46dc BalancedDegradationSettings: add codec specific min bitrate settings.
Bug: none
Change-Id: I7875a3c8499c883ffecad519063c0eb82a8a7073
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/165341
Reviewed-by: Sergey Silkin <ssilkin@webrtc.org>
Commit-Queue: Åsa Persson <asapersson@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30192}
2020-01-09 12:45:33 +00:00
382cc6d8a6 Add incomplete ResourceAdaptationModuleInterface.
This interface will be improved upon iteratively to aid reviewability.
The initial version only handles starting and stopping the module; input
and output of the module is still implementation-specific.

TBR=sprang@webrtc.org

Bug: webrtc:11222
Change-Id: Ie307cfe3d3211c84346c035f2c0e9a632f58221b
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/162580
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Evan Shrubsole <eshr@google.com>
Cr-Commit-Position: refs/heads/master@{#30167}
2020-01-07 13:24:42 +00:00
b08882b625 Refactor out VideoStreamEncoder's overuse logic to separate module.
This CL puts the VideoStreamEncoder's current adaptation logic inside
the new class OveruseFrameDetectorResourceAdaptationModule. The
intention is not to change any behavior, only to move code.

Future CLs should step by step decrease the coupling between
OveruseFrameDetectorResourceAdaptationModule, VideoStreamEncoder and
the VideoStreamEncoder's QualityScaler by introducing more abstract
interfaces. This is not done in this CL because it is large enough as
it is, but the long term goal is to make it possible to replace the
existing overuse module with a different implementation.

This CL relies on existing tests exercising the VideoStreamEncoder, but
part of making overuse logic modular should include testing each module
separately as well as continued integration testing of the
VideoStreamEncoder.

Bug: webrtc:11222
Change-Id: I316a174adfd00d60cdd224a23a5f616efd235d13
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/161953
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Evan Shrubsole <eshr@google.com>
Cr-Commit-Position: refs/heads/master@{#30163}
2020-01-07 09:55:54 +00:00
499de2d395 Fix tiny race condition when Vp9NonFlexMode_* tests were failing.
TSAN complained at each failure, which only added confusion.

Bug: webrtc:11243
Change-Id: Ic36ec13562699478be23050a7b723e1b9bb47f0e
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/164528
Reviewed-by: Henrik Andreassson <henrika@webrtc.org>
Commit-Queue: Yves Gerey <yvesg@google.com>
Cr-Commit-Position: refs/heads/master@{#30161}
2020-01-07 09:29:14 +00:00
c33e4910c5 Prevent 'use after free' by waiting for all queued tasks to be processed.
Sometimes, a task bound to VideoSendStreamTest was called
after the underlying object had been destructed:
 1. |test| goes out of scope.
 2. There might still have been a task in fixture's queue,
    setup by OnSendRtp (capturing [this]) and invoked before
    the destruction of the fixture.

This CL uses the same workaround than BandwidthStatsTest:
block until all posted tasks are processed.

This fixes the following flaky tests:
 * VideoSendStreamTest.ChangingNetworkRoute
 * VideoSendStreamTest.RespectsMinTransmitBitrate*

Bug: webrtc:11156
Change-Id: I229c96d2abbbb60b43e9d9f62ad112507a21fe48
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/163984
Reviewed-by: Niels Moller <nisse@webrtc.org>
Reviewed-by: Karl Wiberg <kwiberg@webrtc.org>
Commit-Queue: Yves Gerey <yvesg@google.com>
Cr-Commit-Position: refs/heads/master@{#30152}
2020-01-07 04:18:51 +00:00
5cad55b240 Signal requested resolution alignment requirements from sinks to sources.
Bug: webrtc:11218
Change-Id: I593b0515ea389bece472234a3c4082ccc5321ea5
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/162400
Commit-Queue: Rasmus Brandt <brandtr@webrtc.org>
Reviewed-by: Karl Wiberg <kwiberg@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30113}
2019-12-19 10:39:04 +00:00
9b540cb553 Correctly process disabled streams in FrameEncodeMetadataWriter
If the first simulcast stream is disabled, but the second one is enabled,
FrameEncodeMetadataWriter would fail to store frame metadata for all
streams and later fail to restore it for encoded frames.

Bug: none
Change-Id: Ib0d257abb863716ea94e56730f7caabef6ebeb64
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/162480
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30106}
2019-12-17 12:28:04 +00:00
cebdbf650d switch RtpVideoStreamReceiver to use VideoRtpDepacketizer interface
instead of creating each time an object with RtpDepacketizer interface

this moves packet payload memcpy from RtpVideoStreamReceiver into
the depacketizers with possibility to remove it from there in follow ups.

Bug: webrtc:11152
Change-Id: If474207eb84d7e9d0207075bd395e60895f0d842
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/162185
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Commit-Queue: Danil Chapovalov <danilchap@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30095}
2019-12-16 12:08:11 +00:00
0f6bf75ab4 Make video engine tests aware of padding packets
Specifically do not try to parse them as rtx packets.

Bug: webrtc:11213, webrtc:11188
Change-Id: I3aa5929af433b1ada9fb26516618d11207f075a5
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/162204
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Yves Gerey <yvesg@google.com>
Commit-Queue: Danil Chapovalov <danilchap@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30094}
2019-12-16 11:43:11 +00:00
b57fe17e7c Migrate video tests and tool to VideoRtpDepacketizer interface
Bug: webrtc:11152
Change-Id: I1e7868ca88b162db8615cb4903bd89d3daac4827
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/161452
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Commit-Queue: Danil Chapovalov <danilchap@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30085}
2019-12-13 11:41:04 +00:00
4fc52c8329 Make struct SynchronizationDelays more general.
Bug: none
Change-Id: Iab263789cc8b51917acb3db2803fa71a927bc62a
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/161640
Reviewed-by: Rasmus Brandt <brandtr@webrtc.org>
Commit-Queue: Åsa Persson <asapersson@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30060}
2019-12-11 13:55:16 +00:00
ef3998ffd1 Add directive to make webrtc metrics optional.
Bug: webrtc:11144
Change-Id: I4e75e6aec033784685de3670e880bb9f2b6ee8d2
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/161043
Reviewed-by: Karl Wiberg <kwiberg@webrtc.org>
Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org>
Commit-Queue: Ying Wang <yinwa@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30040}
2019-12-09 13:55:50 +00:00
33f9d2b383 Migrate WebRTC on FrameGeneratorInterface and remove FrameGenerator class
Bug: webrtc:10138
Change-Id: If85290581a72f81cf60181de7a7134cc9db7716e
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/161327
Reviewed-by: Niels Moller <nisse@webrtc.org>
Reviewed-by: Karl Wiberg <kwiberg@webrtc.org>
Commit-Queue: Artem Titov <titovartem@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30033}
2019-12-07 00:54:26 +00:00