zqz/platform-external-webrtc
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

VideoStreamEncoder: Don't map kNative video frame buffers.

Browse Source 
Follow-up CL to VP8 and VP9 encoders taking care of mapping.
Context again:
  This CL is part of Optimized Scaling efforts. In Chromium, the native
frame buffer is getting an optimized CropAndScale() implementation. To
support HW accelerated scaling, returning pre-scaled images and skipping
unnecessary intermediate downscales, WebRTC needs to 1) use CropAndScale
instead of libyuv::XXXXScale and 2) only map buffers it actually intends
to encode.

In this CL, VideoStreamEncoder no longer calls GetMappedFrameBuffer() on
behalf of the encoders, since the encoders are now able to either do the
mapping or performs ToI420() anyway.

- Tests for old VSE behaviors are updated to test the new behavior (i.e.
  that native frames are pretty much always forwarded).
- The "having to call ToI420() twice" workaround to Android bug
  https://crbug.com/webrtc/12602 is added to H264 and AV1 encoders.

Bug: webrtc:12469
Change-Id: Ibdc2e138d4782a140f433c8330950e61b9829f43
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/211940
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Evan Shrubsole <eshr@google.com>
Cr-Commit-Position: refs/heads/master@{#33548}
This commit is contained in:
Henrik Boström
2021-03-24 09:06:45 +01:00
committed by Commit Bot
parent 5cf8c2c501
commit 56db9ff1e1
5 changed files with 280 additions and 64 deletions
Download Patch File Download Diff File Expand all files Collapse all files

15
modules/video_coding/codecs/av1/libaom_av1_encoder.cc
View File

@ -578,9 +578,22 @@ int32_t LibaomAv1Encoder::Encode(
// Convert input frame to I420, if needed.
VideoFrame prepped_input_frame = frame;
if (prepped_input_frame.video_frame_buffer()->type() !=
VideoFrameBuffer::Type::kI420) {
VideoFrameBuffer::Type::kI420 &&
prepped_input_frame.video_frame_buffer()->type() !=
VideoFrameBuffer::Type::kI420A) {
rtc::scoped_refptr<I420BufferInterface> converted_buffer(
prepped_input_frame.video_frame_buffer()->ToI420());
// The buffer should now be a mapped I420 or I420A format, but some buffer
// implementations incorrectly return the wrong buffer format, such as
// kNative. As a workaround to this, we perform ToI420() a second time.
// TODO(https://crbug.com/webrtc/12602): When Android buffers have a correct
// ToI420() implementaion, remove his workaround.
if (converted_buffer->type() != VideoFrameBuffer::Type::kI420 &&
converted_buffer->type() != VideoFrameBuffer::Type::kI420A) {
converted_buffer = converted_buffer->ToI420();
RTC_CHECK(converted_buffer->type() == VideoFrameBuffer::Type::kI420 ||
converted_buffer->type() == VideoFrameBuffer::Type::kI420A);
}
prepped_input_frame = VideoFrame(converted_buffer, frame.timestamp(),
frame.render_time_ms(), frame.rotation());
}

13
modules/video_coding/codecs/h264/h264_encoder_impl.cc
View File

@ -373,8 +373,19 @@ int32_t H264EncoderImpl::Encode(
return WEBRTC_VIDEO_CODEC_UNINITIALIZED;
}
rtc::scoped_refptr<const I420BufferInterface> frame_buffer =
rtc::scoped_refptr<I420BufferInterface> frame_buffer =
input_frame.video_frame_buffer()->ToI420();
// The buffer should now be a mapped I420 or I420A format, but some buffer
// implementations incorrectly return the wrong buffer format, such as
// kNative. As a workaround to this, we perform ToI420() a second time.
// TODO(https://crbug.com/webrtc/12602): When Android buffers have a correct
// ToI420() implementaion, remove his workaround.
if (frame_buffer->type() != VideoFrameBuffer::Type::kI420 &&
frame_buffer->type() != VideoFrameBuffer::Type::kI420A) {
frame_buffer = frame_buffer->ToI420();
RTC_CHECK(frame_buffer->type() == VideoFrameBuffer::Type::kI420 ||
frame_buffer->type() == VideoFrameBuffer::Type::kI420A);
}
bool send_key_frame = false;
for (size_t i = 0; i < configurations_.size(); ++i) {