This change refactors WgcWindowCapturer into WgcCapturerWin, a source
agnostic capturer, and finishes the implementation to enable both window
and screen capture.
This CL depends on another which must complete first:
196622: Add ability to load CreateDirect3DDeviceFromDXGIDevice from
d3d11.dll | https://webrtc-review.googlesource.com/c/src/+/196622
This feature remains off by default behind a build flag, due to it
adding a depency on the Win10 SDK vv10.0.19041 which not all consumers
of WebRTC have upgraded to. A follow up change later will enable the
rtc_enable_win_wgc build flag, but for now it should remain off.
The basic operation of this class is as follows:
Consumers call either WgcCapturerWin::CreateRawWindowCapturer or
CreateRawScreenCapturer to receive a correctly initialized
WgcCapturerWin object suitable for the desired source type.
Callers then indicate via SelectSource and a SourceId the desired
capture target, and the capturer creates an appropriate WgcCaptureSource
for the correct type (window or screen) using the
WgcCaptureSourceFactory supplied at construction.
Next, callers request frames for the currently selected source and the
capturer then creates a WgcCaptureSession and stores it in a map for
more efficient capture of multiple sources.
The WgcCaptureSession is supplied with a GraphicsCaptureItem created by
the WgcCaptureSource. It uses this item to create a
Direct3D11CaptureFramePool and create and start a
GraphicsCaptureSession.
Once started, captured frames will begin to be deposited into the
FramePool. Typically, one would listen for the FrameArrived event and
process the frame then, but due to the synchronous nature of the
DesktopCapturer interface, and to avoid a more complicated multi-
threaded architecture we ignore the FrameArrived event. Instead, we
wait for a request for a frame from the caller, then we check the
FramePool for a frame, and process it on demand.
Processing a frame involves moving the image data from an
ID3D11Texture2D stored in the GPU into a texture that is accessible
from the CPU, and then copying the data into the new WgcDesktopFrame
class. This copy is necessary as otherwise we would need to manage the
lifetimes of the CaptureFrame and ID3D11Texture2D objects, lest the
buffer be invalidated.
Once we've copied the data and returned it to the caller, we can unmap
the texture and exit the scope of the GetFrame method, which will
destruct the CaptureFrame object. At this point, the CaptureSession
will begin capturing a new frame, and will soon deposit it into the
FramePool and we can repeat.
Bug: webrtc:9273
Change-Id: I02263c4fd587df652b04d5267fad8965330d0f5b
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/200161
Reviewed-by: Jamie Walch <jamiewalch@chromium.org>
Reviewed-by: Guido Urdaneta <guidou@webrtc.org>
Commit-Queue: Austin Orion <auorion@microsoft.com>
Cr-Commit-Position: refs/heads/master@{#33083}
Relanding this after a build break in a downstream consumer caused a
revert. I've removed the include of the windows.graphics.capture.interop
header and instead replaced it with an include of
windows.graphics.directx.direct3d11.h which is where the IDirect3DDevice
data type that we need is declared. Before this was being included
transitively through the WGC header, which was a mistake.
Original change's description:
Add ability to load CreateDirect3DDeviceFromDXGIDevice from d3d11.dll
Creating a Direct3D11Device from a DXGIDevice is necessary for the new
WGC based window capturer. However, the
CreateDirect3DDeviceFromDXGIDevice API is not available on all
versions of Windows, which means we have to load this function from
d3d11.dll at runtime.
You can see how this function will be used in this CL:
196624: Finish implementing WGC Window Capturer and add unit tests. |
https://webrtc-review.googlesource.com/c/src/+/196624
I also ensure we don't leak HSTRINGs in GetActivationFactory and fix
up some includes in ScopedComInitializer.
Bug: webrtc:9273
Change-Id: I56a5eef29815a09297bd2cdad4c5e4265dd7e27e
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/203200
Commit-Queue: Austin Orion <auorion@microsoft.com>
Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#33081}
Reduce amount of dynamic memory used to generate rtcp message
Remove option to request several types of rtcp message as unused
Deduplicated PacketContainer and PacketSender as constructs to create packets
Bug: None
Change-Id: Ib2e20a72a9bd73a441ae6b72a13e18ab5885f5c8
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/203261
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Danil Chapovalov <danilchap@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#33068}
The SpatialIndex value from an EncodedImage is 0-based, but values were
off by 1.
Bug: none
Change-Id: Ie74e6450ddef1cfaee68fa230c441030fa86a64a
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/203525
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Florent Castelli <orphis@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#33067}
In AudioDeviceIOS, when we call Stop() on the VoiceProcessingAudioUnit,
we do not always detach the I/O thread checker in preparation for a new
start. This means that if we start up the VoiceProcessingAudioUnit - and
subsequently a new AURemoteIO thread to deal with I/O operations - the
DCHECK in OnDeliverRecordedData and OnGetPlayoutData will fail. Note
that we want to detach the I/O thread checker regardless of whether
Stop() returns with a success status or not. The success status is
dictated by the iOS function AudioOutputUnitStop. The documentation of
this function does not guarantee that the audio unit will not stop in
the case the function returns with an error code. That is to say, it is
possible the audio unit stops even if the function Stop() returns false.
Therefore, it is safer to prepare the I/O thread checker for a new start
in either case.
Change-Id: Iee50a2457959aff2e6089e9a664c649dc4dbbbd6
Bug: webrtc:12382
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/202945
Reviewed-by: Henrik Andreassson <henrika@webrtc.org>
Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org>
Commit-Queue: Mirko Bonadei <mbonadei@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#33063}
This could potentially lead to unnecessary restarts since it is also
started after the encoder is created. However, it is needed since the
hardware acceleration support can change even though the encoder has
not been recreated.
Bug: b/145730598
Change-Id: Iad1330e7c7bdf769a68c4ecf7abb6abbf3a4fe71
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/203140
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Jakob Ivarsson <jakobi@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#33060}
To add ref counting to any class while avoiding
virtual functions for reference counting.
This template can both slightly reduce binary size
and slightly improve performance.
Bug: webrtc:11308
Change-Id: I90ac735f6c220ee2a1a991a71039acdb0ca86453
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/198845
Commit-Queue: Danil Chapovalov <danilchap@webrtc.org>
Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org>
Reviewed-by: Niels Moller <nisse@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#33058}
To make VideoCodec::scalability_mode the only option to set and
change the scalability structure, for easier maintainability.
Bug: webrtc:11404
Change-Id: I6570e9a93ddf2897ff7584c5d20a246346e853e2
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/192361
Reviewed-by: Philip Eliasson <philipel@webrtc.org>
Commit-Queue: Danil Chapovalov <danilchap@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#33056}
Finding the array element with the largest argmax is a fairly common
operation, so it makes sense to have a Neon optimized version. The
implementation is done by first finding both the min and max value, and
then returning whichever has the largest argmax.
Bug: chromium:12355
Change-Id: I088bd4f7d469b2424a7265de10fffb42764567a1
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/201622
Commit-Queue: Ivo Creusen <ivoc@webrtc.org>
Reviewed-by: Per Åhgren <peah@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#33052}
Using WebRTC-VP9-PerformanceFlags and settings a multi-layer config,
and then configuring the codec in non-svc mode would cause us to not
set the cpu speed in libvpx. For some reason, that could trigger a
crash in the encoder.
This CL fixes that, and adds new test coverage for the code affected
byt the trial.
Bug: chromium:1167353, webrtc:11551
Change-Id: Iddb92fe03fc12bac37717908a8b5df4f3d411bf2
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/202761
Commit-Queue: Erik Språng <sprang@webrtc.org>
Reviewed-by: Danil Chapovalov <danilchap@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#33051}
The race occurs if the transport is being destroyed at the same time as
a callback occurs on the usrsctp timer thread (for example, for a
retransmission). Fixed by slightly extending the scope of mutex
acquisition to include posting a task to the network thread, where it's
safe to do further work.
Bug: chromium:1162424
Change-Id: Ia25c96fa51cd4ba2d8690ba03de8af9e9f1605ea
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/202560
Reviewed-by: Harald Alvestrand <hta@webrtc.org>
Commit-Queue: Taylor <deadbeef@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#33048}
This CL allows separate dump sets to be used when dumping internal
APM data using audioproc_f, opening up for reducing the amount of
data to be dumped.
Bug: webrtc:5298
Change-Id: I8286933ceed10db074f2064414cc08e8b12653fa
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/196089
Commit-Queue: Per Åhgren <peah@webrtc.org>
Reviewed-by: Gustaf Ullberg <gustaf@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#33047}
DependencyDescriptor and vp9 wrapper understand key frame differently
when it comes to the first layer frame with spatial_id>0
This CL adds and use DD's interpretation of the key frame when deciding
if DD should be supported going forward.
Bug: webrtc:11999
Change-Id: I11a809a315e18bd856bb391576c6ea1f427e33be
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/202760
Commit-Queue: Danil Chapovalov <danilchap@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#33046}
To determine the appropriate amount of shifting to prevent overflow in a
cross correlation, it is necessary to have the max value of both
sequences. However, only one was calculated in the ilbc code. This CL
calculates the max of the other sequence and correctly takes both into
account.
Bug: chromium:1161837
Change-Id: I3ba8eee0814bb5eda3769c0ce6caf2681c7525e1
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/202253
Reviewed-by: Sam Zackrisson <saza@webrtc.org>
Commit-Queue: Ivo Creusen <ivoc@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#33043}
Functionality wise, there should be no change with this CL, aside
from updating tests to anticipate OnPacketReceived to handle the packet
asynchronously (as already was the case via BaseChannel).
This only removes the network->worker hop out of the BaseChannel
class into the WebRTC MediaChannel implementations. However, it updates
the interface contract between BaseChannel and MediaChannel to align
with how we want things to work down the line, i.e. avoid hopping to
the worker thread for every rtp packet.
The following steps will be to update the video and voice channel
classes to call Call::DeliverPacket on the network thread and only
handle unsignalled SSRCs on the worker (exception case).
Bug: webrtc:11993
Change-Id: If0540874444565dc93773aee89d862f3bfc9c502
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/202242
Reviewed-by: Harald Alvestrand <hta@webrtc.org>
Commit-Queue: Tommi <tommi@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#33040}
The extmap-allow-mixed SDP attribute signals that one- and two-byte RTP
header extensions can be mixed. In practice, this also means that WebRTC
will support two-byte RTP header extensions when this is signaled by
both peers.
Bug: webrtc:9985
Change-Id: I80a3f97bab162c7d9a5acf2cae07b977641c039d
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/197943
Commit-Queue: Emil Lundmark <lndmrk@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Reviewed-by: Harald Alvestrand <hta@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#33036}
Also making some more state const.
Instances of this class are currently constructed and used on the
"worker thread" but as part of the work for bug webrtc:11993, the
instances will be moved over to the network thread. Since the
class as is does not require synchronization, that is a good property
to make explicit now and then make sure we maintain it in the
transition.
Bug: webrtc:11993
Change-Id: Id587a746ce0a4363b9e9871ae1749549f8ef3e24
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/202681
Commit-Queue: Tommi <tommi@webrtc.org>
Reviewed-by: Niels Moller <nisse@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#33035}
