Support native scaling of VideoFrameBuffers in LibvpxVp9Encoder.

This CL is part of Optimized Scaling efforts. In Chromium, the native
frame buffer is getting an optimized CropAndScale() implementation. To
support HW accelerated scaling, returning pre-scaled images and skipping
unnecessary intermediate downscales, WebRTC needs to 1) use CropAndScale
instead of libyuv::XXXXScale and 2) only map buffers it actually intends
to encode.
- To achieve this, WebRTC encoders are updated to map kNative video
  buffers so that in a follow-up CL VideoStreamEncoder can stop mapping
  intermediate buffer sizes.

In this CL LibvpxVp9Encoder is updated to map kNative buffers of pixel
formats it supports and convert ToI420() if the kNative buffer is
something else. A fake native buffer that keeps track of which
resolutions were mapped, MappableNativeBuffer, is added.

Because VP9 is currently an SVC encoder and not a simulcast encoder, it
does not need to invoke CropAndScale.

This CL also fixes MultiplexEncoderAdapter, but because it simply
forwards frames it only cares about the pixel format when
|supports_augmented_data_| is true so this is the only time we map it.
Because this encoder is not used with kNative in practise, we don't care
to make this path optimal.

Bug: webrtc:12469, chromium:1157072
Change-Id: I74edf85b18eccd0d250776bbade7a6444478efce
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/212580
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Evan Shrubsole <eshr@google.com>
Cr-Commit-Position: refs/heads/master@{#33526}
This commit is contained in:
Henrik Boström
2021-03-22 12:24:30 +01:00
committed by Commit Bot
parent 2ff25db72a
commit bd9e4a95eb
10 changed files with 488 additions and 42 deletions

View File

@ -45,6 +45,12 @@ class AugmentedVideoFrameBuffer : public VideoFrameBuffer {
// Get the I140 Buffer from the underlying frame buffer
rtc::scoped_refptr<I420BufferInterface> ToI420() final;
// Returns GetI420() of the underlying VideoFrameBuffer.
// TODO(hbos): AugmentedVideoFrameBuffer should not return a type (such as
// kI420) without also implementing that type's interface (i.e.
// I420BufferInterface). Either implement all possible Type's interfaces or
// return kNative.
const I420BufferInterface* GetI420() const final;
private:
uint16_t augmenting_data_size_;