DxgiOutputDuplicator objects hold a reference to the last frame that
they succesfully captured by maintaining a reference to the
SharedDesktopFrame that was passed as their target. This is done because
the DirectX capture APIs may fail to provide an update if there has been
no (or no substantial) change since the last capture call was made.
However, the higher levels of this capture stack
(DxgiDuplicatorController and ScreenCapturerWinDirectX), were unaware of
this, and assumed that the caller of CaptureFrame is the only one who
may have held a reference to the frame. Thus, when CaptureFrame is
called, the DirectX screen capturer assumes that the oldest frame in its
queue can be safely reused.
In the steady state, where capture is not being switched between
monitors, this is fine as there are no competing DxgiOutputDuplicators
being run and this assumption mostly holds true (or the frame is being
overwritten only when the DxgiOutputDuplicator is also done holding it).
However, when capture is being rapidly switched between multiple targets
(e.g. to show a preview of each of the available monitors), this can
result in a frame being held by one DxgiOutputDuplicator being passed to
another as a valid target and overwritten. In the common case of only a
single monitor this is essentially the same as steady state capture,
where there are no competing DxgiOutputDuplicator. In the other common
case of two monitors being captured, the fact that the
ScreenCaptureFrameQueue has two frames ends up masking this issue. Since
each monitor is captured in the same order, the same frame ends up
getting passed to each DxgiOutputDuplicator, so no data actually ends
up getting overwritten. In the case of 3 monitors, the 1st and 3rd
monitor end up sharing a frame, which when capture fails on one of them
surfaces as the other monitor being duplicately shown.
This change addresses the issue by ensuring that each screen that the
ScreenCapturerWinDirectX *actually attempts* to capture, gets it's own
FrameQueue, and thus essentially brings us back to the "steady state"
case for each monitor. Note that this does increase memory usage of
capturers that are switched between multiple targets by 2 frames/target
used (and actually attempted to be captured).
Alternatives considered:
DxgiOutputDuplicator makes a copy of the frame, rather than holding
a reference
This was rejected because adding an additional copy for every
capture upon getting a new frame, would expensive and could degrade
performance.
Allow the DxgiOutputDuplicators to "fail" when there has been no update
This would result in either a breaking change to the API for consumers
or would require the ScreenCapturerWinDirectX to track these last
captured frames; which would result in essentially the same approach,
but with less abstraction for re-using the frames.
Bug: chromium:1296228
Change-Id: I5442ec40e9f234046010b562b258db63693ccc6b
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/256043
Reviewed-by: Mark Foltz <mfoltz@chromium.org>
Commit-Queue: Alexander Cooper <alcooper@chromium.org>
Cr-Commit-Position: refs/heads/main@{#36295}
This CL addresses an issue where the desktop appears to freeze after
resizing the desktop in a curtained CRD session when using the DXGI
capturer. This problem does not reproduce when using the GDI capturer
nor does it reproduce when the Windows session is attached to the local
console.
After digging in, it appears that the DXGI DuplicateOutput API stops
providing updated frame data. No errors are returned but yet no data is
produced. The problem is that when in this condition, there isn't a
good way to discern between this problem and a case where the desktop
is actually static.
The DxgiDuplicatorController already contains logic to attempt to
capture a frame prior to returning success after reinitialization. This
logic works fine in the console case and occasionally works in the
detached session case. What I noticed in my reproductions was that DXGI
would produce a few frames before hanging (usually 1-2 but sometimes 3
or 4). My solution is to check the session state and adjust the number
of frames we attempt to capture (I also simplified the wait logic as
there was a bug in the time calc and it seemed more complicated than it
needed to be).
One option considered would be to introduced a new differ class higher
up in the stack which would run the GDI and DXGI capturers in parallel
(instead of in the fallback configuration as they are today) however
that seemed like overkill for this specific issue.
Bug: chromium:1307357
Change-Id: Idba4bb9b2aa7692040344d480be3f0d09b9ce9e9
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/256214
Reviewed-by: Alexander Cooper <alcooper@chromium.org>
Reviewed-by: Jamie Walch <jamiewalch@google.com>
Commit-Queue: Joe Downing <joedow@google.com>
Cr-Commit-Position: refs/heads/main@{#36286}
Change adds callbacks to the class so that the remote desktop portal can
still make use of this class for selecting sources but can provide its
own implementation on what to do after the sources are selected.
Furthermore, few getters are exposed in the class interface so as to
allow the remote desktop portal class to leverage them when sending the
captured pipewire frames onto the capture stream's consumer. Setters are
added for session, pipewire stream node id and few interfaces are made
public since remote desktop portal relies on them (e.g.
`SelectSources`).
The reason behind the change is that remote desktop portal depends on
screen cast portal for selecting sources. Also the setup to select
devices to control remotely as well as source selection should be
handled as part of the same session (and session should be
instantiated only once).
Currently, starting the screencast portal calls into a callback chain
that not only selects the sources but also starts the session but with
this change a consumer, such as remote desktop portal, can hook into
this callback chain by overriding the callbacks and provide a custom
callback chain from there onwards, if need be.
Bug: chromium:1291247
Change-Id: I983aff062ec2ddf52fdef5545fc58fede416e6ed
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/249862
Reviewed-by: Alexander Cooper <alcooper@chromium.org>
Commit-Queue: Salman Malik <salmanmalik@google.com>
Cr-Commit-Position: refs/heads/main@{#36285}
Update `early_execute_margin` after process packets, and the test case.
Original change's description:
>Pacer: Reduce TQ wake up and improve packet size estimation
>
>The TQ Pacer schedules delayed task according to target time of
>PacingController. It drains all valid ProcessPackets() in single loop,
>denies retired scheduled tasks, and round up the timeout to 1ms.
>
>This CL also improves packet size estimation in TQ Pacer by removing
>zero initialization, and introduces `include_overhead_` configuration.
>
>Tests:
>1. webrtc_perf_tests: MaybeProcessPackets() calls
> 2075147 -> 2007995
>
>2. module_unittests: MaybeProcessPackets() calls
> 203393 -> 183563
>
>3. peerconnection_unittests: MaybeProcessPackets() calls
> 66713-> 64333
>
>Bug: webrtc:13417, webrtc:13437
>Change-Id: I18eb0a36dbe063c606b1f27014df74a65ebfc486
>Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/242962
>Reviewed-by: Erik Språng <sprang@webrtc.org>
>Reviewed-by: Henrik Boström <hbos@webrtc.org>
>Commit-Queue: Erik Språng <sprang@webrtc.org>
>Cr-Commit-Position: refs/heads/main@{#36179}
Bug: webrtc:13417, webrtc:13437
Change-Id: I79f2554cf02364b67ce7073698611a3ae337a73b
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/256145
Reviewed-by: Erik Språng <sprang@webrtc.org>
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Owners-Override: Erik Språng <sprang@webrtc.org>
Commit-Queue: Erik Språng <sprang@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#36283}
This reverts commit 865d94e45258c9c8876ea4cbdd5dade510cb7d93.
First patch is the same as original cl. Second patch includes a fix to ensure the clamped bitrate does not increase to 85% of the last network estimate.
Bug: none
Change-Id: Idf1b2af3fb60c0d392c48c1b6c0d8526f900f9d6
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/256016
Reviewed-by: Diep Bui <diepbp@webrtc.org>
Commit-Queue: Per Kjellander <perkj@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#36270}
The diff capturer wrapper doesn't work if the frame doesn't have any
rectangle and a static image is observed while chromoting. This change
adds a rectangle to the frame object, as done by other capturers, and
this in turn ensures that the wrapper that calulcates diffs from one
frame to the next can do its job.
Bug: chromium:1291247
Change-Id: I5bf1981f34b3a88ad4d82a081fed1ce210f71ed0
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/251205
Reviewed-by: Alexander Cooper <alcooper@chromium.org>
Commit-Queue: Salman Malik <salmanmalik@google.com>
Cr-Commit-Position: refs/heads/main@{#36263}
Extract helper methods from screencast portal that can be
reused for remote desktop portal client.
Bug: chromium:1291247
Change-Id: I66d09c75f0c34d81c7ceff8998720fbbd1902ac8
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/249860
Reviewed-by: Alexander Cooper <alcooper@chromium.org>
Commit-Queue: Salman Malik <salmanmalik@google.com>
Cr-Commit-Position: refs/heads/main@{#36256}
This reverts commit e66e6a845b50f212ebb60234446cfc9db897879c.
Reason for revert:
May cause BWE to increase on delay increase if link capacity estimate is too high.
Original change's description:
> Apply lower bound of delay based estimate in AimdRateControl::ClampBitrate
>
> This move the functionality of applying the lower bound of a network estimate to AimdRateControl::ClampBitrate instead of ChangeBitrate.
> The purpose is to be able to also clamp probe estimates set by AimdRateControl::SetEstimate as well.
>
> Bug: none
> Change-Id: I6a4d64d2e98bb99da06010e2edaf20dc42880e37
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/255823
> Reviewed-by: Per Kjellander <perkj@webrtc.org>
> Commit-Queue: Per Kjellander <perkj@webrtc.org>
> Reviewed-by: Diep Bui <diepbp@webrtc.org>
> Cr-Commit-Position: refs/heads/main@{#36219}
Bug: none
Change-Id: I8c65b1461160dbf3d35e50ef2cc6f9bc305c2b15
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/256011
Bot-Commit: rubber-stamper@appspot.gserviceaccount.com <rubber-stamper@appspot.gserviceaccount.com>
Commit-Queue: Per Kjellander <perkj@webrtc.org>
Reviewed-by: Diep Bui <diepbp@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#36248}
Add a flag to override the key pressed state when simulating APM.
The current behavior changes as follows:
- Wav files simulation: instead of simulating continuous key press
events only if the transient suppressor (TS) sub-module is active,
allow to simulate the events regardless of whether TS is used;
the default key pressed state is used if the command line flag is
unspecified, otherwise it is overridden (either always false or
always true)
- AEC dump simulation: instead of simulating continuous key press
events when `--ts 2` is specified, allow to simulate the events
regardless of whether TS is used; the state recorded in the AEC
dump is used if the command line flag is unspecified, otherwise
it is overridden (either always false or always true)
- The `--ts 2` option (continuous key events) is now equivalent to
`--ts 1`.
Bug: webrtc:13663
Change-Id: I5ebe96283db73ee235ec2b2795d91d4e241a3527
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/256003
Reviewed-by: Per Åhgren <peah@webrtc.org>
Commit-Queue: Alessio Bazzica <alessiob@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#36247}
Check whether there are any cursor metadata before we try to validate
and use them, otherwise we might crash on this.
Bug: webrtc:13429
Change-Id: I365da59a189b6b974cebafc94fec49d5b942efae
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/255601
Reviewed-by: Alexander Cooper <alcooper@chromium.org>
Commit-Queue: Alexander Cooper <alcooper@chromium.org>
Cr-Commit-Position: refs/heads/main@{#36240}
- Use NtpTime instead of pair of uint32_t to represent ntp time
- Increase precision estimate with NtpTime precision instead of ms precision
- Hide helper structs as private types
- Modernize interface to prefer return values over output parameters
- embed LinearRegression helper into the only user: UpdateParameters
Bug: webrtc:13757
Change-Id: I0a62a03e2869b2ae1eacaa15253accc43ba0a598
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/254780
Reviewed-by: Niels Moller <nisse@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Danil Chapovalov <danilchap@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#36232}
getDisplayMedia capture the view of the screens and windows
in the capture dialog, but the issue is that captured view
of the Youtube somehow is blank. It repros only in certain
circumstances, for example, Canary channel.
If user reinstall the Canary as fresh new, we observed that
it doesn't repro.
Cause:
We aren't sure what's cause of this one yet.
Solution:
We decided to provide fallback WGC capturer when the main
capturer (GDI) shows blank. WGC could show yellow outline
in prior Win11 OS, but yellow outline looks better than
blank.
The blank detector and fallback capturer are what screen capturer
already supported. So, the solution will follow similar
pattern in the window capturer.
Bug: webrtc:13726
Change-Id: I620c817d259d7bb5c295adab11c4444349ab1c6c
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/252625
Reviewed-by: Alexander Cooper <alcooper@chromium.org>
Commit-Queue: Alexander Cooper <alcooper@chromium.org>
Cr-Commit-Position: refs/heads/main@{#36224}
This is a reland of commit 43a69b3f46059b103c35079744dc09a0e2bb2948
Original change's description:
> Experimentally reduce TaskQueuePacedSender's delayed task precision.
>
> This CL reduces the delayed task precision of non-probes in accordance
> with DD go/slacked-task-queue-paced-sender. The precision is only
> deduced if field trial "WebRTC-SlackedTaskQueuePacedSender" is enabled
> though.
>
> Bug: webrtc:13824
> Change-Id: I37e53b24e343f4f08059be08a3cda74f5484cc05
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/255341
> Reviewed-by: Erik Språng <sprang@webrtc.org>
> Commit-Queue: Henrik Boström <hbos@webrtc.org>
> Cr-Commit-Position: refs/heads/main@{#36214}
Bug: webrtc:13824
Change-Id: I86cace6f4f6bf23d51c75b3d18f8d24fff0f5b74
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/255826
Auto-Submit: Mirko Bonadei <mbonadei@webrtc.org>
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Erik Språng <sprang@webrtc.org>
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#36221}
When congestion window is used, two different mechanisms can currently
update the outstanding data state in the pacer:
* OnPacketSent() withing the pacer itself, when a packet is sent
* UpdateOutstandingData(), when RtpTransportControllerSend either:
a. Receives an OnPacketSent() callback (increase outstanding data)
b. Receives transport feedback (decrease outstanding data)
This creates a lot of calls to UpdateOutstandingData(), more than one
per sent packet. Each requires locking and/or thread jumps. To avoid
that, this CL moves the congestion window state to
RtpTransportController send - and we only post a congested flag down
the the pacer when the state is changed.
The only benefit I can see is of the old way is we prevent sending
new packets immedately when the window is full, rather than in some
edge cases queue extra packets on the network task queue before the
congestion signal is received. That should be rare and benign.
I think this simplified logic, which is easier to read and more
performant, is a better tradeoff.
Bug: webrtc:13417
Change-Id: I326dd88db86dc0d6dc685c61920654ac024e57ef
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/255600
Auto-Submit: Erik Språng <sprang@webrtc.org>
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#36220}
This move the functionality of applying the lower bound of a network estimate to AimdRateControl::ClampBitrate instead of ChangeBitrate.
The purpose is to be able to also clamp probe estimates set by AimdRateControl::SetEstimate as well.
Bug: none
Change-Id: I6a4d64d2e98bb99da06010e2edaf20dc42880e37
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/255823
Reviewed-by: Per Kjellander <perkj@webrtc.org>
Commit-Queue: Per Kjellander <perkj@webrtc.org>
Reviewed-by: Diep Bui <diepbp@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#36219}
Increase the history size of clipping_predictor_evaluator_. Use one-sample
accuracy in clipping detection for the evaluator.
Bug: webrtc:12774
Change-Id: I8c1bbfe69fe55af73ce14992e49ef7295b3ce926
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/241602
Reviewed-by: Alessio Bazzica <alessiob@webrtc.org>
Commit-Queue: Hanna Silen <silen@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#36218}
This CL reduces the delayed task precision of non-probes in accordance
with DD go/slacked-task-queue-paced-sender. The precision is only
deduced if field trial "WebRTC-SlackedTaskQueuePacedSender" is enabled
though.
Bug: webrtc:13824
Change-Id: I37e53b24e343f4f08059be08a3cda74f5484cc05
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/255341
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#36214}
Add field trials to audio api.
It is added as a pointer with nullptr as default.
It is not (yet) used anywhere.
Usage of field trials comes in subsequent patches.
Bug: webrtc:10335
Change-Id: Icbe22d95c356a6fefde34590f11ea63f005ab09e
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/255521
Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org>
Reviewed-by: Harald Alvestrand <hta@webrtc.org>
Commit-Queue: Jonas Oreland <jonaso@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#36213}
The TQ Pacer schedules delayed task according to target time of
PacingController. It drains all valid ProcessPackets() in single loop,
denies retired scheduled tasks, and round up the timeout to 1ms.
This CL also improves packet size estimation in TQ Pacer by removing
zero initialization, and introduces `include_overhead_` configuration.
Tests:
1. webrtc_perf_tests: MaybeProcessPackets() calls
2075147 -> 2007995
2. module_unittests: MaybeProcessPackets() calls
203393 -> 183563
3. peerconnection_unittests: MaybeProcessPackets() calls
66713-> 64333
Bug: webrtc:13417, webrtc:13437
Change-Id: I18eb0a36dbe063c606b1f27014df74a65ebfc486
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/242962
Reviewed-by: Erik Språng <sprang@webrtc.org>
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Commit-Queue: Erik Språng <sprang@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#36179}
This CL fixes the issue where artifacts appear during capture with WGC
when the capture source is resized. A video of the issue is available
here: https://bugs.chromium.org/p/webrtc/issues/detail?id=9273#c44
The solution is to use CopySubresourceRegion instead of CopyResource to
only copy valid data into our texture. Additionally, we moved the call
to CreateMappedTexture to before the call to CopySubresourceRegion, as
the latter requires both textures to be of the same size.
Bug: webrtc:9273
Change-Id: I114458d95cbf58550ff653a985dd84db4741e0f8
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/254100
Reviewed-by: Alexander Cooper <alcooper@chromium.org>
Commit-Queue: Austin Orion <auorion@microsoft.com>
Cr-Commit-Position: refs/heads/main@{#36163}
- add recovered sequence number and length of the recovered packet
- increase level of periodic logging to LS_INFO
- log for every packet on LS_VERBOSE
This makes it easier to validate and debug flexfec implementations.
BUG=None
Change-Id: I6f9e73e72ec3dcc0531f7adc62ac7019c7899270
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/254120
Reviewed-by: Harald Alvestrand <hta@webrtc.org>
Commit-Queue: Philipp Hancke <philipp.hancke@googlemail.com>
Cr-Commit-Position: refs/heads/main@{#36162}
This cl/
1) move WebRtcKeyValueConfig from api/transport to api/ directory.
2) add a test/ScopedKeyValueConfig (compare ScopedFieldTrials).
3) removes usage of webrtc::field_trial:: from the pc/ directory.
4) removes a few unused includes of system_wrappers/field_trial.h.
Bug: webrtc:10335
Change-Id: If29c07900dbe791050b0a5ad05332bedfad035f2
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/253903
Reviewed-by: Harald Alvestrand <hta@webrtc.org>
Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org>
Commit-Queue: Jonas Oreland <jonaso@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#36160}
Additionally,
* Moved to its own GN target.
* Added unittests.
* Removed unused variable `_zeroWallClock`.
* Renamed variables to match style guide.
* Moved fields _dTS and _wrapArounds to variables.
Change-Id: I7aa8b8dec55abab49ceabe838dabf2a7e13d685d
Bug: webrtc:13756
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/253580
Reviewed-by: Niels Moller <nisse@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Reviewed-by: Harald Alvestrand <hta@webrtc.org>
Commit-Queue: Evan Shrubsole <eshr@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#36147}
As stashed frames are retried their `tl0_pic_idx` are again unwrapped which can lead to the `tl0_unwrapper_` to unwrap the `tl0_pic_idx` of newer frames backwards. Instead unwrap the `tl0_pid_idx` only once and save it with the frame if necessary.
In this CL
- Only unwrap the TL0 once in ManageFrame.
- Split ManageFrameInternal into ManageFrameFlexible and ManageFrameGof.
- Save the unwrapped TL0 with the stashed frame.
Bug: none
Change-Id: I56e6b071c0082682e010c049c537d66060635567
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/253844
Reviewed-by: Danil Chapovalov <danilchap@webrtc.org>
Commit-Queue: Philip Eliasson <philipel@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#36146}
* Uses DataSize to represent incoming and outgoing bytes.
* Puts units into doubles as they enter the Kalman filter
* Moved to its own GN target.
Change-Id: I1e7d5486a00a7158d418f553a6c77f9dd56bf3c2
Bug: webrtc:13756
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/253121
Reviewed-by: Åsa Persson <asapersson@webrtc.org>
Commit-Queue: Evan Shrubsole <eshr@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#36143}
