zqz/platform-external-webrtc
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Communicate encoder resolutions via rtc::VideoSinkWants.

Browse Source 
This will allow us to optimize the internal buffers of
webrtc::VideoFrame for the resolution(s) that we actually want to
encode.

Bug: webrtc:12469, chromium:1157072
Change-Id: If378b52b5e35aa9a9800c1f7dfe189437ce43253
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/208540
Reviewed-by: Niels Moller <nisse@webrtc.org>
Reviewed-by: Harald Alvestrand <hta@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#33342}
This commit is contained in:
Henrik Boström
2021-02-25 10:30:39 +01:00
committed by Commit Bot
parent bb52bdf095
commit 1124ed1ab2
6 changed files with 208 additions and 5 deletions
Download Patch File Download Diff File Expand all files Collapse all files

36
api/video/video_source_interface.h
View File

@ -12,6 +12,7 @@
#define API_VIDEO_VIDEO_SOURCE_INTERFACE_H_
#include <limits>
#include <vector>
#include "absl/types/optional.h"
#include "api/video/video_sink_interface.h"
@ -22,6 +23,15 @@ namespace rtc {
// VideoSinkWants is used for notifying the source of properties a video frame
// should have when it is delivered to a certain sink.
struct RTC_EXPORT VideoSinkWants {
struct FrameSize {
FrameSize(int width, int height) : width(width), height(height) {}
FrameSize(const FrameSize&) = default;
~FrameSize() = default;
int width;
int height;
};
VideoSinkWants();
VideoSinkWants(const VideoSinkWants&);
~VideoSinkWants();
@ -49,8 +59,34 @@ struct RTC_EXPORT VideoSinkWants {
// Note that this field is unrelated to any horizontal or vertical stride
// requirements the encoder has on the incoming video frame buffers.
int resolution_alignment = 1;
// The resolutions that sink is configured to consume. If the sink is an
// encoder this is what the encoder is configured to encode. In singlecast we
// only encode one resolution, but in simulcast and SVC this can mean multiple
// resolutions per frame.
//
// The sink is always configured to consume a subset of the
// webrtc::VideoFrame's resolution. In the case of encoding, we usually encode
// at webrtc::VideoFrame's resolution but this may not always be the case due
// to scaleResolutionDownBy or turning off simulcast or SVC layers.
//
// For example, we may capture at 720p and due to adaptation (e.g. applying
// |max_pixel_count| constraints) create webrtc::VideoFrames of size 480p, but
// if we do scaleResolutionDownBy:2 then the only resolution we end up
// encoding is 240p. In this case we still need to provide webrtc::VideoFrames
// of size 480p but we can optimize internal buffers for 240p, avoiding
// downsampling to 480p if possible.
//
// Note that the |resolutions| can change while frames are in flight and
// should only be used as a hint when constructing the webrtc::VideoFrame.
std::vector<FrameSize> resolutions;
};
inline bool operator==(const VideoSinkWants::FrameSize& a,
const VideoSinkWants::FrameSize& b) {
return a.width == b.width && a.height == b.height;
}
template <typename VideoFrameT>
class VideoSourceInterface {
public: