The encoder can drop all frames for extended periods if it has produced over budget. After 2 seconds without any encoded frames, the video send stream times out and deallocates the stream.
Ideally the send stream should keep track if frames are captured instead of encoded, but keeping the stream alive using OnDroppedFrame can work as a proxy for that.
Bug: webrtc:11062
Change-Id: Id7ec1ff333427643453c4a36d1db03ca826cd9ea
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/158700
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Jakob Ivarsson <jakobi@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#29662}
For the automatic content type detection we need to know if the update
rect is trusted or just not available.
Currently we only care if it's not empty, so in case of no update rect
available, full frame resolution was set as a changed region.
This CL makes the update_rect field optional but should be a no-op in the
current code, as absence of update_rect is treated as a full update via
a new getter method |update_rect_or_full_frame()|.
Bug: webrtc:11058
Change-Id: I913545b71ac2fc845861549ac34eb1b630012109
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/158673
Commit-Queue: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Niels Moller <nisse@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#29654}
- Use minimum start bitrate to drop frame and adapt resolution in the
beginning of call.
- Use minimum bitrate to decide whether or not resolution should be
increased based on quality in MAINTAIN_FRAMERATE and BALANCED modes.
In BALANCED mode bitrate limits provided by the corresponding field
trial are prioritized over the limits provided by encoder.
Bug: webrtc:10853
Change-Id: I8257eb64565bcafa6ae9887a1af18e90f8400cac
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/156302
Commit-Queue: Sergey Silkin <ssilkin@webrtc.org>
Reviewed-by: Åsa Persson <asapersson@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#29461}
This is a reland of 809198edfff416fce8d75b574a43afab5e67b1cd
A fix was made in https://webrtc-review.googlesource.com/c/src/+/154343
which fixed the regression issues caused by the original patch.
Original change's description:
> Fix minor regression caused by a8336d3
>
> VideoEncoder::SetRates was being called unnessesarily when the fields
> appended to RateControlParameters were changed. Since SetRates only
> cares about RateControlParameters, it should have only been called if
> the RateControlParameters themselves were actually changed.
>
> Bug: webrtc:10126
> Change-Id: Ic47d67e642a3043307fec950e5fba970d9f95167
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/152829
> Reviewed-by: Erik Språng <sprang@webrtc.org>
> Commit-Queue: Evan Shrubsole <eshr@google.com>
> Cr-Commit-Position: refs/heads/master@{#29208}
Bug: webrtc:10126
Change-Id: Iecc3ab6a5cd1193a1fa8e824dcf4f0b8165f9bf8
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/154359
Commit-Queue: Evan Shrubsole <eshr@google.com>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#29356}
Lately there were 2 separate bugs, where seeing this information in the
log could help immediately figuring out the problem.
Bug: none
Change-Id: I3f2b2d5864106cdb231715e1702edee3b9b05caa
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/154566
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#29338}
Without invalidation the codec configuration may not get updated. This
is prone to happen when the bandwidth is below the min or above the max
in which case the last_encoder_rate_settings_ may have not changed.
This led to a regression in b6a45dd, where last_encoder_rate_settings_
would not change the allocation or frame rate, but the frame rate would
have been set by the video adapter. Thus the frame rates were set
incorrectly, leading to lower values in the regression tests.
I have re-run this scenario against some of the metric drops and the
regression appears to be fixed.
Bug: webrtc:10126
Change-Id: I0fa6c9b71e7aff5dd80e53119db109d97eed98b6
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/154343
Commit-Queue: Evan Shrubsole <eshr@google.com>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#29301}
This reverts commit 809198edfff416fce8d75b574a43afab5e67b1cd.
Reason for revert: Performance regressions that need to be addressed.
Original change's description:
> Fix minor regression caused by a8336d3
>
> VideoEncoder::SetRates was being called unnessesarily when the fields
> appended to RateControlParameters were changed. Since SetRates only
> cares about RateControlParameters, it should have only been called if
> the RateControlParameters themselves were actually changed.
>
> Bug: webrtc:10126
> Change-Id: Ic47d67e642a3043307fec950e5fba970d9f95167
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/152829
> Reviewed-by: Erik Språng <sprang@webrtc.org>
> Commit-Queue: Evan Shrubsole <eshr@google.com>
> Cr-Commit-Position: refs/heads/master@{#29208}
TBR=sprang@webrtc.org,eshr@google.com
# Not skipping CQ checks because original CL landed > 1 day ago.
Bug: webrtc:10126
Change-Id: I133cbe5d8cb894ed944ae8a2d0f63a78bbed72ee
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/153484
Commit-Queue: Erik Språng <sprang@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#29221}
VideoEncoder::SetRates was being called unnessesarily when the fields
appended to RateControlParameters were changed. Since SetRates only
cares about RateControlParameters, it should have only been called if
the RateControlParameters themselves were actually changed.
Bug: webrtc:10126
Change-Id: Ic47d67e642a3043307fec950e5fba970d9f95167
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/152829
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Evan Shrubsole <eshr@google.com>
Cr-Commit-Position: refs/heads/master@{#29208}
In this CL:
- Renamed EncoderFailureCallback to EncoderSwitchRequestCallback. An encoder
switch request can now also be made with a configuration that specifies which
codec/implementation to switch to.
- Added "WebRTC-NetworkCondition-EncoderSwitch" field trial that specifies
switching conditions and desired codec to switch to.
- Added checks to trigger the switch based on these conditions.
Bug: webrtc:10795
Change-Id: I9d3a9a39a7c4827915a40bdceed10b581d70b90a
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/151900
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Niels Moller <nisse@webrtc.org>
Commit-Queue: Philip Eliasson <philipel@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#29196}
The stable target rate is used to make smarter choices in the rate
to chose which layers to enable in SVC or simulcast modes.
the addition of hysteresis, we can improve a call quality by reducing
the amount of resolution switch.
Bug: webrtc:10126
Change-Id: I04d0df9e6bbe247e2f2a668207ff74d475e2464c
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/150642
Reviewed-by: Niels Moller <nisse@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Florent Castelli <orphis@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#29112}
The VideoBitrateAllocator subclasses may actually allocate more than the
target, in order to satisfy the min bitrate constraint. In this case,
make sure the bandwidth allocation we signal to the encoder is at least
this large.
Bug: chromium:995462
Change-Id: I08b89a7c54392330d773e13c1b0a3eff42f81672
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/151125
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Commit-Queue: Erik Språng <sprang@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#29040}
Add separate setting for configuring min bitrate that only applies when
adapting up in resolution.
Bug: none
Change-Id: I83d33ac3110a22602065b8d83130e3f619cb1eba
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/150329
Commit-Queue: Åsa Persson <asapersson@webrtc.org>
Reviewed-by: Mirta Dvornicic <mirtad@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#28970}
If a framerate reduction (input fps - restricted fps) is less than the
configured diff, shorten interval to next qp check.
Bug: none
Change-Id: Ia0b9e0638e5ba75cdc20a1bb45bfcb7d858c5f89
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/149040
Commit-Queue: Åsa Persson <asapersson@webrtc.org>
Reviewed-by: Magnus Flodman <mflodman@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#28880}
Add possibility to configure min bitrate based on resolution.
Only adapt up if bw estimate is above the min bitrate for next higher resolution.
Bug: none
Change-Id: Ie38faae07d23336675ec33697ace6f6fed322efa
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/148598
Reviewed-by: Magnus Flodman <mflodman@webrtc.org>
Commit-Queue: Åsa Persson <asapersson@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#28863}
Also:
- DCHECK that bitrate limits recommended by encoder are reasonable.
- Restrict target bitrate such that it never exceed the max bitrate.
Bug: webrtc:10853
Change-Id: Ie43d30a7acfc8fa115deffd94165844248ce7945
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/148442
Commit-Queue: Sergey Silkin <ssilkin@webrtc.org>
Reviewed-by: Åsa Persson <asapersson@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#28822}
optional<int> initial_bitrate_interval_ms: time interval since start of call
where fast adapt down is allowed.
optional<double> initial_bitrate_factor: try fast adapt down if bw estimate is
below initial bitrate * factor.
Bug: none
Change-Id: I63e1fdaac6556d8e9a961a42e11c925f9ecb9771
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/147725
Reviewed-by: Sergey Silkin <ssilkin@webrtc.org>
Commit-Queue: Åsa Persson <asapersson@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#28753}
We want to evaluate more data in order to make better choices in the
bitrate allocators.
In order to freely update the parameter list without
breaking the API many times for projects customizing them, we'll use a
struct instead.
Bug: webrtc:10126
Change-Id: I443f86781c5134950294cdd1e3197a47447cf973
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/141418
Commit-Queue: Florent Castelli <orphis@webrtc.org>
Reviewed-by: Tommi <tommi@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#28748}
If VideoEncoderConfig::max_bitrate_bps is unset then max bitrate of
video stream is set equal to max bitrate value recommended by encoder
for given resolution via encoder capabilities (if available).
Bug: webrtc:10796
Change-Id: I7fce9afc476b794a16956e694e891faee110048e
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/144526
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Sergey Silkin <ssilkin@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#28515}
In this CL:
- Added WEBRTC_VIDEO_CODEC_ENCODER_FAILURE return code that can
be returned by the encoder wrapper in case of a broken encoder.
- Added EncoderFailureCallback interface that can be called
to request encoder fallback to be performed. Implemented by
WebRtcVideoChannel and called from the VideoStreamEncoder.
- Updated SelectSendVideoCodec to select all compatible codecs instead
of just one.
Bug: webrtc:10795
Change-Id: I87a83fd02e48c40493c930471c06c3d0941031ab
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/140888
Commit-Queue: Philip Eliasson <philipel@webrtc.org>
Reviewed-by: Magnus Jedvert <magjed@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#28462}
Before this change, an attempt to recreate video encoder would fail if
video encoder factory supports only single instance of an encoder.
Added tracking of max number of existed simultaneously encoder
instances to VideoEncoderProxyFactory.
Bug: webrtc:10776
Change-Id: I317cbdf1af94dfb4c72bf99c5cd4ce7b454188fa
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/144044
Commit-Queue: Sergey Silkin <ssilkin@webrtc.org>
Reviewed-by: Åsa Persson <asapersson@webrtc.org>
Reviewed-by: Niels Moller <nisse@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#28457}
- Don't reset encoder if max/min bitrate changed.
- Removed min/max bitrate DCHECKs from encoder wrappers.
- Reset encoder if start_bitrate changed. Only do this if encoding
has not yet started.
- Updated ReconfigureBitratesSetsEncoderBitratesCorrectly test.
- Removed EncoderSetupPropagatesCommonEncoderConfigValues test since it
was a subset of ReconfigureBitratesSetsEncoderBitratesCorrectly.
Bug: webrtc:10773
Change-Id: Id9cbb2ea229232fd95967819e2a937b26948de9f
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/144028
Commit-Queue: Sergey Silkin <ssilkin@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Åsa Persson <asapersson@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#28446}
The purpose of this interface is to allow VideoEncoder to override
the bandwidth allocation set by FecController in RtpVideoSender.
This CL defines the interface and sends it down to VideoSender.
Two upcoming CLs will:
1. Make LibvpxVp8Encoder pass it on to the (injectable)
FrameBufferController, where it might be put to good use.
2. Modify RtpVideoSender to respond to the message sent to it
via this API.
TBR=kwiberg@webrtc.org
Bug: webrtc:10769
Change-Id: I2ef82f0ddcde7fd078e32d8aabf6efe43e0f7f8a
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/143962
Commit-Queue: Elad Alon <eladalon@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Reviewed-by: Rasmus Brandt <brandtr@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#28416}
Some downstream clients have custom frame types that can't be converted.
The rest of EncodeVideoFrame is protected against these frames, but the
crop code assumes ToI420 always succeeds.
Bug: None
Change-Id: I8f4279e3975d3ae8cd1da59f7e84fafe0404fd15
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/141646
Commit-Queue: Noah Richards <noahric@chromium.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#28256}
Before this change the max bitrate could be updated after it was passed
to rate allocator.
Bug: none
Change-Id: I742fca0f122bef3e95c1a768d6e844f8c28b6279
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/141661
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Sergey Silkin <ssilkin@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#28253}
This is a reland of 11dfff0878c949f2e19d95a0ddc209cdad94b3b4
Now that I am sure that WebRTC code is not calling the obsolete
versions, I will just remove the NOT_REACHED and call the
new version from the old ones, so as not to trip up downstream
projects.
Original change's description:
> Inform VideoEncoder of negotiated capabilities
>
> After this CL lands, an announcement will be made to
> discuss-webrtc about the deprecation of one version
> of InitEncode().
>
> Bug: webrtc:10720
> Change-Id: Ib992af0272bbb16ae16ef7e69491f365702d179e
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/140884
> Reviewed-by: Karl Wiberg <kwiberg@webrtc.org>
> Reviewed-by: Sami Kalliomäki <sakal@webrtc.org>
> Reviewed-by: Erik Språng <sprang@webrtc.org>
> Commit-Queue: Elad Alon <eladalon@webrtc.org>
> Cr-Commit-Position: refs/heads/master@{#28224}
TBR=sakal@webrtc.org,kwiberg@webrtc.org,sprang@webrtc.org
Bug: webrtc:10720
Change-Id: I46c69e45c190805c07f7e51acbe277d7eebd1600
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/141412
Commit-Queue: Elad Alon <eladalon@webrtc.org>
Reviewed-by: Elad Alon <eladalon@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#28236}
In several places VideoFrame::Builder is used to create a new VideoFrame
when intent is to change only one or two fields of a const VideoFrame&.
This approach is bad because each and every metadata field have to be
added to all the places.
Instead, this CL adds missing setters and refactors the code to use
full copy of a VideoFrame and update required fields only.
Along the way few actual bugs are fixed, e.g. when ColorSpace isn't copied
when frame rotation or buffer is cropped or converted.
Bug: webrtc:10460
Change-Id: I2895a473ca938b150eed2916c689060bdf58cb25
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/140102
Reviewed-by: Niels Moller <nisse@webrtc.org>
Commit-Queue: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#28170}
Reland with fixes.
Currently some video frames metadata like rotation or ntp timestamps are
copied in every encoder and decoder separately. This CL makes copying to
happen at a single place for send or receive side. This will make it
easier to add new metadata in the future.
Also, added some missing tests.
Original Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/133346
Bug: webrtc:10460
Change-Id: I98629589fa55ca1d74056033cf86faccfdf848cd
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/136582
Commit-Queue: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Danil Chapovalov <danilchap@webrtc.org>
Reviewed-by: Niels Moller <nisse@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#27930}
This reverts commit 4fb12b0caec9faa57cfbceb0f86b0e10c32a0cc2.
Reason for revert: Breaks some asan chromium bots
Original change's description:
> Reland "Copy video frames metadata between encoded and plain frames in one place"
>
> Reland with fixes.
>
> Currently some video frames metadata like rotation or ntp timestamps are
> copied in every encoder and decoder separately. This CL makes copying to
> happen at a single place for send or receive side. This will make it
> easier to add new metadata in the future.
>
> Also, added some missing tests.
>
> Original Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/133346
>
> Bug: webrtc:10460
> Change-Id: Ia71198685de7fbd990704b575231cdce94dc0645
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/134961
> Reviewed-by: Johannes Kron <kron@webrtc.org>
> Reviewed-by: Niels Moller <nisse@webrtc.org>
> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
> Commit-Queue: Ilya Nikolaevskiy <ilnik@webrtc.org>
> Cr-Commit-Position: refs/heads/master@{#27828}
TBR=ilnik@webrtc.org,nisse@webrtc.org,sprang@webrtc.org,kron@webrtc.org,artit@webrtc.org
# Not skipping CQ checks because original CL landed > 1 day ago.
Bug: webrtc:10460
Change-Id: I9c87a43a716622b389974cb8377f973573fc29a5
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/135747
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Commit-Queue: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#27895}
Make Vp8FrameBufferController::UpdateConfiguration return a set
of desired overrides. These overrides are cumulative with
previously returned override sets.
Bug: webrtc:10382
Change-Id: I1aa9544ae0cf6c57115e80963b3bbcdc3101db5e
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/134649
Reviewed-by: Rasmus Brandt <brandtr@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Elad Alon <eladalon@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#27835}
Reland with fixes.
Currently some video frames metadata like rotation or ntp timestamps are
copied in every encoder and decoder separately. This CL makes copying to
happen at a single place for send or receive side. This will make it
easier to add new metadata in the future.
Also, added some missing tests.
Original Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/133346
Bug: webrtc:10460
Change-Id: Ia71198685de7fbd990704b575231cdce94dc0645
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/134961
Reviewed-by: Johannes Kron <kron@webrtc.org>
Reviewed-by: Niels Moller <nisse@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Commit-Queue: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#27828}
