It would be nice for this to stay in video stream encoder,
but this feature is mostly related to quality scaling. Perhaps
something easier to understand is possible in the future.
Bug: webrtc:11222
Change-Id: I71705f33ff94bbcf2fb9b5c94226c8e76dcba94c
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/168051
Commit-Queue: Evan Shrubsole <eshr@google.com>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30446}
This allows for further refactoring, eventually moving
all of quality scaler out of video stream encoder.
Bug: webrtc:11222
Change-Id: Id121608da56f57549a616ccc5f141bb598668b40
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/167728
Commit-Queue: Evan Shrubsole <eshr@google.com>
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30417}
This removes the last remaining explicit reference from
OveruseFrameDetectorResourceAdaptationModule to
VideoStreamEncoder.
VideoStreamEncoder's call to SetEncoderSettings() inside
ReconfigureEncoder() is moved a few lines down - it was discovered that
during these lines the EncoderInfo config could get modified in
response to InitEncode() - so this fixes a potential bug.
Bug: webrtc:11222
Change-Id: I9746f28a4df8e631e297669c10636bf17b39acec
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/167363
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Evan Shrubsole <eshr@google.com>
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30381}
This defines the following methods:
- OnFrame(), replaces SetLastFramePixelCount().
- OnFrameDroppedDueToSize(), a rename of FrameDroppedDueToSize() to
match the other methods.
- OnEncodeStarted(), a rename of the incorrectly named FrameCaptured().
- OnEncodeCompleted(), a rename of the poorly named FrameSent().
In order to get rid of SetLastFramePixelCount(), the "we don't know the
frame size" use case - which was previously implicitly avoided by
invoking SetLastFramePixelCount() with a made-up value for
last_frame_info_ - is now avoided using ".value_or()" in
LastInputFrameSizeOrDefault(). This does mean that a constant 144p
resolution value is referenced in two places, but the fact that this is
a magic value is at least made explicit. This may help future
improvements.
Bug: webrtc:11222
Change-Id: I3b28daa8c5ecf57c6537957d4759f15e24bb2234
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/166961
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Evan Shrubsole <eshr@google.com>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30352}
All uses of encoded_image are const, except for the copy for running on
the encoder_queue_.
Bug: None
Change-Id: I7fc8cb46f6afb42a2d27961d3d3ff8d9e63fe1b5
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/166442
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Evan Shrubsole <eshr@google.com>
Cr-Commit-Position: refs/heads/master@{#30351}
The poorly named SetEncoderStartBitrate() is renamed
SetEncoderTargetBitrate() and added to the abstract resource adaptation
module interface.
The so-called "start bitrate" was updated to match the target bitrate,
so this was only ever a "start bitrate" until we had any estimates. The
variable is renamed in VideoStreamEncoder as well, and usage of optional
types are introduced to avoid magical values in a few places in the
existing code.
Bug: webrtc:11222
Change-Id: Idde92f68f34616aa3c34ab77a791fdbe7ea7af26
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/166880
Reviewed-by: Evan Shrubsole <eshr@google.com>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30347}
This moves SetHasInputVideoAndDegradationPreference() to the encoder
queue. OveruseFrameDetectorResourceAdaptationModule is now entirely
single-threaded, including its inner class VideoSourceRestrictor.
VideoStreamEncoder now protects the module with RTC_GUARDED_BY. This
ensures it is safely used, even without a SequenceChecker inside of the
module. The module's |encoder_queue_| is removed.
The one task queue reference that is needed - passing down the current
task queue to StartCheckForOveruse() - is replaced by a TaskQueueBase*
(instead of rtc::TaskQueue*), enabling obtaining the current queue with
TaskQueueBase::Current(). (There is no rtc::TaskQueue::Current().)
Furthermore, the only uses of VideoSourceSinkController that isn't on
the encoder queue are documented, with a TODO saying if these are moved
the VideoSourceSinkController could also be made single-threaded.
However since this requires introducing a delay to
VideoStreamEncoder::SetSource() and VideoStreamEncoder::Stop(),
arguably a more risky change, if this is to be attempted that should be
in a separate CL.
Bug: webrtc:11222
Change-Id: I448ca5125708d5f66b95b0b180d6d24cc356dfa9
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/165783
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Evan Shrubsole <eshr@google.com>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30263}
This is part of the work for making VideoStreamEncoder responsible for
configuring its source/sink and limiting the responsibility of
OveruseFrameDetectorResourceAdaptationModule to only output relevant
VideoSourceRestrictions.
BEFORE THIS CL
Prior to this CL, OveruseFrameDetector was responsible for performing
AddOrUpdateSink() on the source, which it did using its nested class
VideoSourceProxy.
AddOrUpdateSink() could happen for both adaptation and non-adaptation
related reasons. For example:
- Adaptation related: AdaptUp() or AdaptDown() happens, causing updated
VideoSourceRestrictions.
- Non-adaptation related: VideoStreamEncoder asks the module to
reconfigure the source/sink for it, such as with
SetMaxFramerateAndAlignment() or SetWantsRotationApplied().
AFTER THIS CL
AddOrUpdateSink() is performed by VideoSourceController, which is owned
by VideoStreamEncoder. Any reconfiguration has to go through the
VideoStreamEncoder. This means that:
- Non-adaptation related settings happen between VideoStreamEncoder and
VideoSourceController directly (without going through the adaptation
module).
- Adaptation related changes can be expressed in terms of
VideoSourceRestrictions. OveruseFrameDetectorResourceAdaptationModule
only has to output the restrictions and not know or care about other
source/sink settings.
For now, VideoSourceController has to know about DegradationPreference.
In a future CL, the DegradationPreference logic should move back to
the adaptation module. The VideoSourceRestrictions are fully capable of
expressing all possible source/sink values without the "modifier" that
is the degradation preference.
Bug: webrtc:11222
Change-Id: I0f058c4700ca108e2d9f212e38b61f6f728aa419
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/162802
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Evan Shrubsole <eshr@google.com>
Cr-Commit-Position: refs/heads/master@{#30228}
The VideoSourceRestrictions describe the maximum pixels per frame and
max frame rate of a video source.
This CL makes the ResourceAdaptationModuleInterface responsible for the
reconfiguration of video sources. The VideoSourceRestrictions is the
output of an adaptation module, and the ResourceAdaptationModuleListener
handles the callback for when the source restrictions change.
The OveruseFrameDetectorResourceAdaptationModule is updated to output
its changes using these interfaces, and VideoStreamEncoder - now a
listener - is made responsible for triggering the reconfiguring the
video source.
Performing the reconfiguration still requires interacting with the
VideoSourceProxy - it is still partially responsible for keeping track
of rtc::VideoSinkWants settings and performing AddOrUpdateSink(). For
now this may look a bit weird: the VideoSourceProxy tells the
VideoStreamEncoder about the new restrictions, and then the
VideoStreamEncoder tells the VideoSourceProxy to apply these
restrictions on the source/sink. This exercises the listener though, and
unblocks the next CL.
The next CL should move all "configuring the source" logic to the
VideoStreamEncoder instead, so that the only information that is tracked
by OveruseFrameDetectorResourceAdaptationModule is what it actually
outputs to the listener. See the next CL
(https://webrtc-review.googlesource.com/c/src/+/162802) where a
VideoSourceController is introduced, to be owned by the
VideoStreamEncoder rather than the adaptation module.
Bug: webrtc:11222
Change-Id: I450ce74f51d96c4b98009a06134db671893d8fdc
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/162522
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Evan Shrubsole <eshr@google.com>
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30227}
This interface will be improved upon iteratively to aid reviewability.
The initial version only handles starting and stopping the module; input
and output of the module is still implementation-specific.
TBR=sprang@webrtc.org
Bug: webrtc:11222
Change-Id: Ie307cfe3d3211c84346c035f2c0e9a632f58221b
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/162580
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Evan Shrubsole <eshr@google.com>
Cr-Commit-Position: refs/heads/master@{#30167}
This CL puts the VideoStreamEncoder's current adaptation logic inside
the new class OveruseFrameDetectorResourceAdaptationModule. The
intention is not to change any behavior, only to move code.
Future CLs should step by step decrease the coupling between
OveruseFrameDetectorResourceAdaptationModule, VideoStreamEncoder and
the VideoStreamEncoder's QualityScaler by introducing more abstract
interfaces. This is not done in this CL because it is large enough as
it is, but the long term goal is to make it possible to replace the
existing overuse module with a different implementation.
This CL relies on existing tests exercising the VideoStreamEncoder, but
part of making overuse logic modular should include testing each module
separately as well as continued integration testing of the
VideoStreamEncoder.
Bug: webrtc:11222
Change-Id: I316a174adfd00d60cdd224a23a5f616efd235d13
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/161953
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Evan Shrubsole <eshr@google.com>
Cr-Commit-Position: refs/heads/master@{#30163}
If WebRTC-AutomaticAnimationDetectionScreenshare experiment is enabled,
content type is screenshare and degradation preference is BALANCED,
then input resolution is restricted if update_rect of the incoming frames
is the same for considerable amount of time and is big enough.
This entails treating BALANCED degradation preference for screenshare as
MAINTAIN_RESOLUTION in adaptation logic.
Bug: webrtc:11058
Change-Id: I903dddf53fcbd7c8eac6c5b1447225b15fd8fe5f
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/161097
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30002}
For the automatic content type detection we need to know if the update
rect is trusted or just not available.
Currently we only care if it's not empty, so in case of no update rect
available, full frame resolution was set as a changed region.
This CL makes the update_rect field optional but should be a no-op in the
current code, as absence of update_rect is treated as a full update via
a new getter method |update_rect_or_full_frame()|.
Bug: webrtc:11058
Change-Id: I913545b71ac2fc845861549ac34eb1b630012109
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/158673
Commit-Queue: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Niels Moller <nisse@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#29654}
- Use minimum start bitrate to drop frame and adapt resolution in the
beginning of call.
- Use minimum bitrate to decide whether or not resolution should be
increased based on quality in MAINTAIN_FRAMERATE and BALANCED modes.
In BALANCED mode bitrate limits provided by the corresponding field
trial are prioritized over the limits provided by encoder.
Bug: webrtc:10853
Change-Id: I8257eb64565bcafa6ae9887a1af18e90f8400cac
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/156302
Commit-Queue: Sergey Silkin <ssilkin@webrtc.org>
Reviewed-by: Åsa Persson <asapersson@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#29461}
This is a reland of 809198edfff416fce8d75b574a43afab5e67b1cd
A fix was made in https://webrtc-review.googlesource.com/c/src/+/154343
which fixed the regression issues caused by the original patch.
Original change's description:
> Fix minor regression caused by a8336d3
>
> VideoEncoder::SetRates was being called unnessesarily when the fields
> appended to RateControlParameters were changed. Since SetRates only
> cares about RateControlParameters, it should have only been called if
> the RateControlParameters themselves were actually changed.
>
> Bug: webrtc:10126
> Change-Id: Ic47d67e642a3043307fec950e5fba970d9f95167
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/152829
> Reviewed-by: Erik Språng <sprang@webrtc.org>
> Commit-Queue: Evan Shrubsole <eshr@google.com>
> Cr-Commit-Position: refs/heads/master@{#29208}
Bug: webrtc:10126
Change-Id: Iecc3ab6a5cd1193a1fa8e824dcf4f0b8165f9bf8
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/154359
Commit-Queue: Evan Shrubsole <eshr@google.com>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#29356}
This reverts commit 809198edfff416fce8d75b574a43afab5e67b1cd.
Reason for revert: Performance regressions that need to be addressed.
Original change's description:
> Fix minor regression caused by a8336d3
>
> VideoEncoder::SetRates was being called unnessesarily when the fields
> appended to RateControlParameters were changed. Since SetRates only
> cares about RateControlParameters, it should have only been called if
> the RateControlParameters themselves were actually changed.
>
> Bug: webrtc:10126
> Change-Id: Ic47d67e642a3043307fec950e5fba970d9f95167
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/152829
> Reviewed-by: Erik Språng <sprang@webrtc.org>
> Commit-Queue: Evan Shrubsole <eshr@google.com>
> Cr-Commit-Position: refs/heads/master@{#29208}
TBR=sprang@webrtc.org,eshr@google.com
# Not skipping CQ checks because original CL landed > 1 day ago.
Bug: webrtc:10126
Change-Id: I133cbe5d8cb894ed944ae8a2d0f63a78bbed72ee
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/153484
Commit-Queue: Erik Språng <sprang@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#29221}
VideoEncoder::SetRates was being called unnessesarily when the fields
appended to RateControlParameters were changed. Since SetRates only
cares about RateControlParameters, it should have only been called if
the RateControlParameters themselves were actually changed.
Bug: webrtc:10126
Change-Id: Ic47d67e642a3043307fec950e5fba970d9f95167
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/152829
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Evan Shrubsole <eshr@google.com>
Cr-Commit-Position: refs/heads/master@{#29208}
In this CL:
- Renamed EncoderFailureCallback to EncoderSwitchRequestCallback. An encoder
switch request can now also be made with a configuration that specifies which
codec/implementation to switch to.
- Added "WebRTC-NetworkCondition-EncoderSwitch" field trial that specifies
switching conditions and desired codec to switch to.
- Added checks to trigger the switch based on these conditions.
Bug: webrtc:10795
Change-Id: I9d3a9a39a7c4827915a40bdceed10b581d70b90a
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/151900
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Niels Moller <nisse@webrtc.org>
Commit-Queue: Philip Eliasson <philipel@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#29196}
The stable target rate is used to make smarter choices in the rate
to chose which layers to enable in SVC or simulcast modes.
the addition of hysteresis, we can improve a call quality by reducing
the amount of resolution switch.
Bug: webrtc:10126
Change-Id: I04d0df9e6bbe247e2f2a668207ff74d475e2464c
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/150642
Reviewed-by: Niels Moller <nisse@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Florent Castelli <orphis@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#29112}
If a framerate reduction (input fps - restricted fps) is less than the
configured diff, shorten interval to next qp check.
Bug: none
Change-Id: Ia0b9e0638e5ba75cdc20a1bb45bfcb7d858c5f89
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/149040
Commit-Queue: Åsa Persson <asapersson@webrtc.org>
Reviewed-by: Magnus Flodman <mflodman@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#28880}
optional<int> initial_bitrate_interval_ms: time interval since start of call
where fast adapt down is allowed.
optional<double> initial_bitrate_factor: try fast adapt down if bw estimate is
below initial bitrate * factor.
Bug: none
Change-Id: I63e1fdaac6556d8e9a961a42e11c925f9ecb9771
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/147725
Reviewed-by: Sergey Silkin <ssilkin@webrtc.org>
Commit-Queue: Åsa Persson <asapersson@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#28753}
If VideoEncoderConfig::max_bitrate_bps is unset then max bitrate of
video stream is set equal to max bitrate value recommended by encoder
for given resolution via encoder capabilities (if available).
Bug: webrtc:10796
Change-Id: I7fce9afc476b794a16956e694e891faee110048e
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/144526
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Sergey Silkin <ssilkin@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#28515}
In this CL:
- Added WEBRTC_VIDEO_CODEC_ENCODER_FAILURE return code that can
be returned by the encoder wrapper in case of a broken encoder.
- Added EncoderFailureCallback interface that can be called
to request encoder fallback to be performed. Implemented by
WebRtcVideoChannel and called from the VideoStreamEncoder.
- Updated SelectSendVideoCodec to select all compatible codecs instead
of just one.
Bug: webrtc:10795
Change-Id: I87a83fd02e48c40493c930471c06c3d0941031ab
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/140888
Commit-Queue: Philip Eliasson <philipel@webrtc.org>
Reviewed-by: Magnus Jedvert <magjed@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#28462}
- Don't reset encoder if max/min bitrate changed.
- Removed min/max bitrate DCHECKs from encoder wrappers.
- Reset encoder if start_bitrate changed. Only do this if encoding
has not yet started.
- Updated ReconfigureBitratesSetsEncoderBitratesCorrectly test.
- Removed EncoderSetupPropagatesCommonEncoderConfigValues test since it
was a subset of ReconfigureBitratesSetsEncoderBitratesCorrectly.
Bug: webrtc:10773
Change-Id: Id9cbb2ea229232fd95967819e2a937b26948de9f
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/144028
Commit-Queue: Sergey Silkin <ssilkin@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Åsa Persson <asapersson@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#28446}
The purpose of this interface is to allow VideoEncoder to override
the bandwidth allocation set by FecController in RtpVideoSender.
This CL defines the interface and sends it down to VideoSender.
Two upcoming CLs will:
1. Make LibvpxVp8Encoder pass it on to the (injectable)
FrameBufferController, where it might be put to good use.
2. Modify RtpVideoSender to respond to the message sent to it
via this API.
TBR=kwiberg@webrtc.org
Bug: webrtc:10769
Change-Id: I2ef82f0ddcde7fd078e32d8aabf6efe43e0f7f8a
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/143962
Commit-Queue: Elad Alon <eladalon@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Reviewed-by: Rasmus Brandt <brandtr@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#28416}
Reland with fixes.
Currently some video frames metadata like rotation or ntp timestamps are
copied in every encoder and decoder separately. This CL makes copying to
happen at a single place for send or receive side. This will make it
easier to add new metadata in the future.
Also, added some missing tests.
Original Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/133346
Bug: webrtc:10460
Change-Id: I98629589fa55ca1d74056033cf86faccfdf848cd
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/136582
Commit-Queue: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Danil Chapovalov <danilchap@webrtc.org>
Reviewed-by: Niels Moller <nisse@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#27930}
This reverts commit 4fb12b0caec9faa57cfbceb0f86b0e10c32a0cc2.
Reason for revert: Breaks some asan chromium bots
Original change's description:
> Reland "Copy video frames metadata between encoded and plain frames in one place"
>
> Reland with fixes.
>
> Currently some video frames metadata like rotation or ntp timestamps are
> copied in every encoder and decoder separately. This CL makes copying to
> happen at a single place for send or receive side. This will make it
> easier to add new metadata in the future.
>
> Also, added some missing tests.
>
> Original Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/133346
>
> Bug: webrtc:10460
> Change-Id: Ia71198685de7fbd990704b575231cdce94dc0645
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/134961
> Reviewed-by: Johannes Kron <kron@webrtc.org>
> Reviewed-by: Niels Moller <nisse@webrtc.org>
> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
> Commit-Queue: Ilya Nikolaevskiy <ilnik@webrtc.org>
> Cr-Commit-Position: refs/heads/master@{#27828}
TBR=ilnik@webrtc.org,nisse@webrtc.org,sprang@webrtc.org,kron@webrtc.org,artit@webrtc.org
# Not skipping CQ checks because original CL landed > 1 day ago.
Bug: webrtc:10460
Change-Id: I9c87a43a716622b389974cb8377f973573fc29a5
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/135747
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Commit-Queue: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#27895}
Reland with fixes.
Currently some video frames metadata like rotation or ntp timestamps are
copied in every encoder and decoder separately. This CL makes copying to
happen at a single place for send or receive side. This will make it
easier to add new metadata in the future.
Also, added some missing tests.
Original Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/133346
Bug: webrtc:10460
Change-Id: Ia71198685de7fbd990704b575231cdce94dc0645
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/134961
Reviewed-by: Johannes Kron <kron@webrtc.org>
Reviewed-by: Niels Moller <nisse@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Commit-Queue: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#27828}
In this CL:
- Assign frame IDs so that simulcast streams share one frame ID space.
- Added a CodecBufferUsage class that represent how a particular buffer
was used (updated, referenced or both).
- Calculate frame dependencies based on the CodecBufferUsage information.
Bug: webrtc:10342
Change-Id: I4ed5ad703f9376a7d995c04bb757c7d214865ddb
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/131287
Reviewed-by: Niels Moller <nisse@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Danil Chapovalov <danilchap@webrtc.org>
Commit-Queue: Philip Eliasson <philipel@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#27784}
This reverts commit c9a2c5e93aa51606916e6728454bcff26bb75f79.
Reason for revert: Breaks downstream test
Original change's description:
> Reland "Copy video frames metadata between encoded and plain frames in one place"
>
> Reland with fixes: Do not remove extra metadata copies in software decoders as some downstream projects assumes these fields are copied by the encoders.
>
> Currently some video frames metadata like rotation or ntp timestamps are
> copied in every encoder and decoder separately. This CL makes copying to
> happen at a single place for send or receive side. This will make it
> easier to add new metadata in the future.
>
> Also, added some missing tests.
>
> Original Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/133346
>
> Bug: webrtc:10460
> Change-Id: I8e49589bf75f406e2b5ddee34882de0faedbd09a
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/134102
> Commit-Queue: Ilya Nikolaevskiy <ilnik@webrtc.org>
> Reviewed-by: Johannes Kron <kron@webrtc.org>
> Reviewed-by: Erik Språng <sprang@webrtc.org>
> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
> Cr-Commit-Position: refs/heads/master@{#27756}
TBR=ilnik@webrtc.org,sprang@webrtc.org,kron@webrtc.org,artit@webrtc.org
Change-Id: I34cc563ec6383735c2a76a6f45a72a7726b74421
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: webrtc:10460
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/134204
Reviewed-by: Artem Titarenko <artit@google.com>
Commit-Queue: Artem Titarenko <artit@google.com>
Cr-Commit-Position: refs/heads/master@{#27765}
Reland with fixes: Do not remove extra metadata copies in software decoders as some downstream projects assumes these fields are copied by the encoders.
Currently some video frames metadata like rotation or ntp timestamps are
copied in every encoder and decoder separately. This CL makes copying to
happen at a single place for send or receive side. This will make it
easier to add new metadata in the future.
Also, added some missing tests.
Original Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/133346
Bug: webrtc:10460
Change-Id: I8e49589bf75f406e2b5ddee34882de0faedbd09a
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/134102
Commit-Queue: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Johannes Kron <kron@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#27756}
This reverts commit 00d0a0a1a9520fb4323d7e3a1c02133b7b942978.
Reason for revert: Breaks downstream tests
Original change's description:
> Copy video frames metadata between encoded and plain frames in one place
>
> Currently some video frames metadata like rotation or ntp timestamps are
> copied in every encoder and decoder separately. This CL makes copying to
> happen at a single place for send or receive side. This will make it
> easier to add new metadata in the future.
>
> Also, added some missing tests.
>
> Bug: webrtc:10460
> Change-Id: Ia49072c3041e75433f125a61050d2982b2bec1da
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/133346
> Commit-Queue: Ilya Nikolaevskiy <ilnik@webrtc.org>
> Reviewed-by: Johannes Kron <kron@webrtc.org>
> Reviewed-by: Erik Språng <sprang@webrtc.org>
> Cr-Commit-Position: refs/heads/master@{#27719}
TBR=ilnik@webrtc.org,sprang@webrtc.org,kron@webrtc.org
Change-Id: I8960a6cc15e552925129ba0037f197ff3fd93c25
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: webrtc:10460
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/134100
Reviewed-by: Artem Titarenko <artit@webrtc.org>
Commit-Queue: Artem Titarenko <artit@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#27737}
Currently some video frames metadata like rotation or ntp timestamps are
copied in every encoder and decoder separately. This CL makes copying to
happen at a single place for send or receive side. This will make it
easier to add new metadata in the future.
Also, added some missing tests.
Bug: webrtc:10460
Change-Id: Ia49072c3041e75433f125a61050d2982b2bec1da
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/133346
Commit-Queue: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Johannes Kron <kron@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#27719}
Translate LossNotification RTCP messages (sequence number to
timestamp and additional information), then send the translted
message onwards to the encoder.
Bug: webrtc:10501
Change-Id: If2fd943f75c36cf813a83120318d8eefc8c595d2
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/131950
Commit-Queue: Elad Alon <eladalon@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Reviewed-by: Niels Moller <nisse@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#27545}
This CL wires up the new SetRates() method of the video encoders, and
refactors a few things in the process:
Most notably, the VideoStreamEncoderInterface is update so that the
|target_headroom| parameter is replaced with |link_allocation|, meaning
that instead of indicating bitrate capacity in excess of the target
bitrate, it indicates to total network capacity allocated for the
stream including the target bitrate. This matches the VideoEncoder API.
The VideoEncoder::RateControlParameters struct gets a few new helper
methods.
In VideoStreamEncoder, instead of adding more fields to the
|last_observed_bitrate*| family, uses an optional struct that
inherits from VideoEncoder::RateControlParameters.
Bug: webrtc:10481
Change-Id: Iee3965531142ae9b964ed86c0d51db59b1cdd61c
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/131123
Commit-Queue: Erik Språng <sprang@webrtc.org>
Reviewed-by: Niels Moller <nisse@webrtc.org>
Reviewed-by: Åsa Persson <asapersson@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#27487}
Fix issue where RTCP XR target bitrate could be incorrect when temporal layers are configured but it is not supported by encoder.
Bug: webrtc:10475
Change-Id: Ib525eb5f0ad8392e88d2579930ac8e459a1d194b
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/128778
Commit-Queue: Åsa Persson <asapersson@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Reviewed-by: Rasmus Brandt <brandtr@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#27268}
This cl deprecates the FrameType enum, and adds aliases AudioFrameType
and VideoFrameType.
After downstream usage is updated, the enums will be separated
and be moved out of common_types.h.
Bug: webrtc:6883
Change-Id: I2aaf660169da45f22574b4cbb16aea8522cc07a6
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/123184
Commit-Queue: Niels Moller <nisse@webrtc.org>
Reviewed-by: Karl Wiberg <kwiberg@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#27011}
This CL plumbs an additional signal from VideoSendStream down to
VideoStreamEncoder, namely the amount of headroom that's left between
the encoder max bitrate and the current bitrate allocation for the
media track.
This will be used in follow-up CLs to tune encoder rate adjustment
and some codec specific paramaters a bit differently, based on the
knowledge if we are network constrained or not.
Bug: webrtc:10155
Change-Id: Ic6ccc79be5c6845468bab65b4ca9918b56923fa4
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/125981
Reviewed-by: Niels Moller <nisse@webrtc.org>
Commit-Queue: Erik Språng <sprang@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#27008}
This CL takes a few parts of VCMEncodedFrameCallback and
VCMGenericEncoder and folds some aspect directly into
VideoStreamEncoder. Parts related to timing frames are extracted
into a new class FrameEncodeTimer that explicitly handles that.
Bug: webrtc:10164
Change-Id: I9b26f734473b659e4093c84c09fb0ed441290e40
Reviewed-on: https://webrtc-review.googlesource.com/c/124122
Commit-Queue: Erik Språng <sprang@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#26862}
