This reverts commit 1b8ef63876ebfa55a51c8ca9b1d8206bf8233e01.
Reason for revert: Breaks downstream projects. b/155256727
Original change's description:
> Add an optional override for AudioRecord device
>
> This is important when we have multiple named devices connected over
> USB (eg. "Webcam", "Microphone", "Headset") and there is some way to
> choose a specific input device to route from.
>
> Bug: b/154440591
> Change-Id: I8dc1801a5e4db7f7bb439e855d43897c1f7d8bc4
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/173748
> Commit-Queue: Robin Lee <rgl@google.com>
> Reviewed-by: Sami Kalliomäki <sakal@webrtc.org>
> Reviewed-by: Henrik Andreassson <henrika@webrtc.org>
> Cr-Commit-Position: refs/heads/master@{#31130}
TBR=henrika@webrtc.org,sakal@webrtc.org,rgl@google.com
# Not skipping CQ checks because original CL landed > 1 day ago.
Bug: b/154440591, b/155256727
Change-Id: I6836676096d47d9da5702a40b9d127569ad50dda
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/175008
Reviewed-by: Henrik Andreassson <henrika@webrtc.org>
Reviewed-by: Henrik Lundin <henrik.lundin@webrtc.org>
Commit-Queue: Henrik Lundin <henrik.lundin@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#31238}
Now we only await a previous adaptation if the degradataion
preference is the same as our current degradation preference.
Without this guard we can get stuck as detailed in the attached bug.
Bug: webrtc:11562
Change-Id: I91be48546446ef8d01fe901bc6889201a5b97ba6
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/174805
Commit-Queue: Evan Shrubsole <eshr@google.com>
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#31236}
This CL unblocks future Call-Level Mitigation strategies by moving the
ResourceAdaptationProcessor to a separate task queue. This signifies a
major milestone in the new resource adaptation architecture because
with this CL the threading model is in place and moving the Processor
to the Call and increasing its responsibilities is made possible.
In this CL, we still have one Processor per VideoStreamEncoder and the
VideoStreamEncoder is responsible for the creation and the destruction
of its Processor and that Processor's task queue. But the PostTasks are
in place and the decision-making is executed on a separate queue.
This CL:
- Moves ResourceAdaptationProcessor to an adaptation task queue.
It continues to be entirely single-threaded, but now operates on a
separate task queue.
- Makes Resources thread-safe: Interaction with the Processor, i.e.
OnResourceUsageStateMeasured() and IsAdaptationUpAllowed(), happens
on the adaptation task queue. State updates are pushed from the
encoder task queue with PostTasks.
- QualityScalerResource operates on both task queues; the QP usage
callbacks are invoked asynchronously.
- The VideoStreamEncoderResourceManager operates on the encoder task
queue with the following exceptions:
1) Its resources are accessible on any thread (using a mutex). This
is OK because resources are reference counted and thread safe.
This aids adding and removing resources to the Processor on the
adaptation task queue.
2) |active_counts_| is moved to the adaptation task queue. This makes
it possible for PreventAdaptUpDueToActiveCounts to run
IsAdaptationUpAllowed() on the adaptation task queue.
A side-effect of this is that some stats reporting now happen on
the adaptation task queue, but that is OK because
VideoStreamEncoderObserver is thread-safe.
The Manager is updated to take the new threading model into account:
- OnFrameDroppedDueToSize() posts to the adaptation task queue to
invoke the Processor.
- OnVideoSourceRestrictionsUpdated(), now invoked on the adaptation
task queue, updates |active_counts_| synchronously but posts to the
encoder task queue to update video source restrictions (which it
only uses to calculate target frame rate).
- MaybePerformQualityRampupExperiment() posts to the adaptation task
queue to maybe reset video source restrictions on the Processor.
|quality_rampup_done_| is made std::atomic.
Bug: webrtc:11542, webrtc:11520
Change-Id: I1cfd76e0cd42f006a6d2527f5aa2aeb5266ba6d6
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/174441
Reviewed-by: Evan Shrubsole <eshr@google.com>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#31231}
When Chromium displays the selection dialog for screens it gets the thumbnails by calling SelectSource for the first monitor then CaptureFrame, then SelectSource for the next monitor then CaptureFrame, and so on. With 1 or 2 screens this does not show any issues, but with 3 or more screens the program may crash.
The queue of frame buffers is actually just 2 frame buffers that get swapped every time a frame is captured. When you have one monitor both buffers will be sized for it's resolution. When you have two monitor the first buffer is sized for the first monitor and the second buffer for the second monitor. Since the monitors are selected in turn monitors and frame buffers stay matched up and things work fine. With a third monitor the first buffer is sized for the first monitor, but then later reused to capture the third monitor. If the resolution of the third monitor does not match the first we either crash or have extra junk in the frame from when we captured the first monitor.
Bug: chromium:396091
Change-Id: I7b5ee914b02fee48c09422cee1e320396c9550c7
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/174520
Commit-Queue: Jamie Walch <jamiewalch@chromium.org>
Reviewed-by: Jamie Walch <jamiewalch@chromium.org>
Cr-Commit-Position: refs/heads/master@{#31229}
This allows users to inject the residual echo detector, as a step toward making it an optional part of compilation.
Bug: webrtc:11292, webrtc:11539
Change-Id: I7fcc8dbaced67a82851cd6cdcbc115eb01c21fcf
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/174040
Reviewed-by: Per Åhgren <peah@webrtc.org>
Reviewed-by: Karl Wiberg <kwiberg@webrtc.org>
Commit-Queue: Sam Zackrisson <saza@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#31222}
This CL makes the VideoStreamEncoderResourceManager's inner Resources
(PreventAdaptUpDueToActiveCounts,
PreventIncreaseResolutionDueToBitrateResource and
PreventAdaptUpInBalancedResource) not directly depend on any of the
manager's states that will continue to live on the encoder task queue
when the adaptation task queue is introduced in the next CL.
PreventAdaptUpDueToActiveCounts depends on effective degradation
preference, which it can get from the Processor, and the active counts,
which will move to the adaptation queue and is safe to use.
PreventIncreaseResolutionDueToBitrateResource depends on encoder
settings and target bitrate. This Resource now listens to these states
being updated, which may be implemented with a PostTask when the
adaptation queue is added.
PreventAdaptUpInBalancedResource depends on the effective degradation
preference, which it can get from the Processor; balanced settings,
which is a const readonly struct (thread-safe); and encoder target
bitrate, which it listens for being updated (to be PostTask'ed).
All resources depends on GetReasonFromResource() which will be callable
from the adaptation queue.
Bug: webrtc:11542, webrtc:11520
Change-Id: Ifa7bd87d9d8729988073f78f6a37c6f3b8aa4db1
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/174807
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Evan Shrubsole <eshr@google.com>
Cr-Commit-Position: refs/heads/master@{#31220}
This CL adds a queue for pending QualityScalerQpUsageHandlerCallbacks
and private methods for "Queueing", "Handling" and "Aborting" them,
using a sequence number as an ID to ensure we don't accidentally invoke
the same callback twice.
Because we don't have the adaptation task queue yet, callbacks are still
synchronously handled, which means the "pending callbacks" queue would
never have more than 1 element. However, when the adaptation task queue
is added and this is made asynchronous, it will be possible for multiple
callbacks to be pending simultaneously. This design is future-proof.
This CL is split out to aid reviewability. The CL that adds the
adaptation task queue will affect a lot of code. By landing this
separately, the adaptation queue CL will be easier to review.
This CL adds quality_scaler_resource_unittest.cc.
Bug: webrtc:11542, webrtc:11520
Change-Id: I00e7f6bfda9f8e8e82ec25916aa48e9349c8d70c
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/174802
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Evan Shrubsole <eshr@google.com>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#31219}
In a future CL, adaptation processing and stream encoder resource
management will happen on different task queues. When this is the case,
asynchronous tasks will be posted in both directions and some resources
will have internal states used on multiple threads.
This CL makes the Resource class reference counted in order to support
posting tasks to a different threads without risk of use-after-free
when a posted task is executed with a delay. This is preferred over
WeakPtr strategies because WeakPtrs are single-threaded and preferred
over raw pointer usage because the reference counted approach enables
more compile-time and run-time assurance. This is also "future proof";
when resources can be injected through public APIs, ownership needs to
be shared between libwebrtc and the application (e.g. Chrome).
To reduce the risk of making mistakes in the future CL, sequence
checkers and task queue DCHECKs are added as well as other DCHECKs to
make sure things have been cleaned up before destruction, e.g:
- Processor gets a sequence checker. It is entirely single-threaded.
- Processor must not have any attached listeners or resources on
destruction.
- Resources must not have any listeners on destruction.
- The Manager, EncodeUsageResource and QualityScalerResource DCHECKs
they are running on the encoder queue.
- TODOs are added illustrating where we want to add PostTasks in the
future CL.
Lastly, upon VideoStreamEncoder::Stop() we delete the
ResourceAdaptationProcessor. Because the Processor is already used in
posted tasks, some if statements are added to ensure the Processor is
not used after destruction.
Bug: webrtc:11542, webrtc:11520
Change-Id: Ibaa8a61d86d87a71f477d1075a117c28d9d2d285
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/174760
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Evan Shrubsole <eshr@google.com>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#31217}
This reverts commit 6b9c60b06d04bc519195fca1f621b10accfeb46b.
Reason for revert: Breaks downstream test
Original change's description:
> Removes lock release in PacedSender callback.
>
> The PacedSender currently has logic to temporarily release its internal
> lock while sending or asking for padding.
> This creates some tricky situations in the pacing controller where we
> need to consider if some thread can enter while we the process thread is
> actually processing, just temporarily busy sending.
>
> Since the pacing call stack is no longer cyclic, we can actually remove
> this lock-release now.
>
> Bug: webrtc:10809
> Change-Id: Ic59c605252bed1f96a03406c908a30cd1012f995
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/173592
> Reviewed-by: Sebastian Jansson <srte@webrtc.org>
> Commit-Queue: Erik Språng <sprang@webrtc.org>
> Cr-Commit-Position: refs/heads/master@{#31206}
TBR=sprang@webrtc.org,srte@webrtc.org
Change-Id: Ic84eee6097528d0792e3b1f90f36bc78447a0d81
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: webrtc:10809
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/174820
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Erik Språng <sprang@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#31209}
This CL changes the way that AecDumps are created in APM. Instead
of being injected, they are now created via the API.
This removes the AecDumpFactory from the API surface of APM and
makes the API more explicit.
The CL will be followed by one more CL that deprecates the usage
of the AttachAecDump API also within the audio_processing
and the fuzzer folders.
The CL also moves the aec_dump.* files from the include folder
to the aec_dump folder and changes the build files. The reasons
for this are that
1) The content of aec_dump.h is not really part of the API
surface of APM.
2) Those files anyway needed to be moved to a separate build-
target to avoid a circular build-file dependency caused by
the other changes in this CL
Bug: webrtc:5298
Change-Id: I7dd6b49de76eb44158472874e1d4ae17dca9be54
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/174750
Commit-Queue: Per Åhgren <peah@webrtc.org>
Reviewed-by: Sam Zackrisson <saza@webrtc.org>
Reviewed-by: Artem Titov <titovartem@webrtc.org>
Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#31207}
The PacedSender currently has logic to temporarily release its internal
lock while sending or asking for padding.
This creates some tricky situations in the pacing controller where we
need to consider if some thread can enter while we the process thread is
actually processing, just temporarily busy sending.
Since the pacing call stack is no longer cyclic, we can actually remove
this lock-release now.
Bug: webrtc:10809
Change-Id: Ic59c605252bed1f96a03406c908a30cd1012f995
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/173592
Reviewed-by: Sebastian Jansson <srte@webrtc.org>
Commit-Queue: Erik Språng <sprang@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#31206}
For various reasons is_desktop_linux is true on Chromecast builds though
arguably it should not be. This means that the detection logic previously
used is incorrect for Chromecast builds. Since Chromecast needs to
start enabling use_sysroot, this logic needs to explicitly exclude
is_chromecast.
Bug: b/154635846
Change-Id: I6ced6f7e4c78f9d8d7055018e68090883b9e21bd
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/174620
Commit-Queue: Mirko Bonadei <mbonadei@webrtc.org>
Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#31205}
This reduces locking on the decoder thread and moves all stats
management to the worker thread, which also avoids contention between
querying for these stats and the threads where the media processing happens..
Bug: webrtc:11489,webrtc:11490
Change-Id: I802577eab6b48edcbe124c02a1b793a640b74181
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/174205
Commit-Queue: Tommi <tommi@webrtc.org>
Reviewed-by: Rasmus Brandt <brandtr@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#31202}
This offloads the decoder thread with managing histograms,
moves the management over to the thread on which they're queried.
This will allow us to remove more locking from the decoder threads
and avoid contention when querying for stats.
Bug: webrtc:11489
Change-Id: I563c90a0ed01e0b3598ee314d8118622216a2e0f
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/174201
Commit-Queue: Tommi <tommi@webrtc.org>
Reviewed-by: Rasmus Brandt <brandtr@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#31201}
Remove dependency on ProcessThread.
Instead RtpStreamsSynchronizer uses the worker thread
and makes callbacks on the same thread. That in turn
simplifies locking for VideoReceiveStream2, which we'll
take advantage of later.
Bug: webrtc:11489
Change-Id: Id9a5a7977771b92e420a09cc472cfb43de5627cc
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/174221
Commit-Queue: Tommi <tommi@webrtc.org>
Reviewed-by: Magnus Flodman <mflodman@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#31200}
This reverts commit c623495fd1ff90aada0eb625af91ec17843fefd0.
Reason for revert: Need to look into failure in remoting_unittests in Chrome (Webrtc/ConnectionTest.SecondCaptureFailed/0). It looks like the order FrameBuffer2 calls into VCMTiming while receiving frames and updating playout delay values, needs to be synchronized better.
Original change's description:
> Remove playout delay lock.
> Now update the playout delay and related stats on the worker thread.
>
> This was previously reviewed here:
> https://webrtc-review.googlesource.com/c/src/+/172929/
>
> With the exception of reducing unnecessarily broad
> lock scope in one function in rtp_rtcp_impl.cc
> and added comments in rtp_streams_synchronizer.h
>
> Bug: webrtc:11489
> Change-Id: I77807b5da2accfe774255d9409542d358f288993
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/174200
> Commit-Queue: Tommi <tommi@webrtc.org>
> Reviewed-by: Erik Språng <sprang@webrtc.org>
> Cr-Commit-Position: refs/heads/master@{#31193}
TBR=tommi@webrtc.org,sprang@webrtc.org
# Not skipping CQ checks because original CL landed > 1 day ago.
Bug: webrtc:11489
Change-Id: I9149025d2fc10686314e6d4e89d1b92125650c36
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/174757
Reviewed-by: Tommi <tommi@webrtc.org>
Commit-Queue: Tommi <tommi@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#31197}
Modernise functions to unified MOCK_METHOD macro,
delete few deprecated functions on the way.
add one missing function (in MockEncodedImageCallback)
Remove proxy mock function (in MockVideoBitrateAllocatorFactory)
Remove default constructors and destructors
Bug: None
Change-Id: Ibebb0d9e3c9be5877649af7bde8b87222ddf04fb
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/174751
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Sebastian Jansson <srte@webrtc.org>
Reviewed-by: Karl Wiberg <kwiberg@webrtc.org>
Commit-Queue: Danil Chapovalov <danilchap@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#31195}
Now update the playout delay and related stats on the worker thread.
This was previously reviewed here:
https://webrtc-review.googlesource.com/c/src/+/172929/
With the exception of reducing unnecessarily broad
lock scope in one function in rtp_rtcp_impl.cc
and added comments in rtp_streams_synchronizer.h
Bug: webrtc:11489
Change-Id: I77807b5da2accfe774255d9409542d358f288993
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/174200
Commit-Queue: Tommi <tommi@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#31193}
Max encoder bitrate in WebRTC and OpenH264 are different settings. In
WebRTC it is a cap for encoder target bitrate whilst in OpenH264 it is
a peak bitrate. I.e. OpenH264 is allowed to produce bitrate up to
iMaxBitrate for short time interval. That is not what WebRTC expects.
https://webrtc.googlesource.com/src/+/5ee6967c4edc667688d736c27db6f2e7be00dd0a
disabled encoders re-initialization on min/max bitrate change. Reinit of
some HW encoders takes hundreds of milliseconds and causes video freeze.
I missed that max bitrate is used by OpenH264. This caused regression
described in webrtc:11543.
This change sets iMaxBitrate=UNSPECIFIED_BIT_RATE (which is the default
value). Settings iMaxBitrate=UNSPECIFIED_BIT_RATE disables the frame
dropping logic based on that parameter. But the encoder still will drop
frames based on buffer fullness, https://source.chromium.org/chromium/chromium/src/+/master:third_party/openh264/src/codec/encoder/core/src/ratectl.cpp;l=806-807
Bug: webrtc:10773, webrtc:11543
Change-Id: I728be49e0df8a0d9a8f4438299e4c7b4c1497a78
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/174745
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Sergey Silkin <ssilkin@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#31192}
Modernise function to unified MOCK_METHOD macro, delete few deprecated functions on the way.
Remove default constructors to stress they do nothing special
Bug: None
Change-Id: Ie126f38f0589acb65886f25f754ca575c17af29b
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/174583
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Danil Chapovalov <danilchap@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#31191}
while helpful by itself, it is also a preparation
for adding unittests for (to be added) svc features of the encoder.
Bug: webrtc:11404
Change-Id: I62b0645f44579f21f228d406a206b4c01d80dd02
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/174580
Commit-Queue: Danil Chapovalov <danilchap@webrtc.org>
Reviewed-by: Philip Eliasson <philipel@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#31189}
