Commit Graph

321 Commits

Author SHA1 Message Date
bd06b76e5b VideoStreamEncoder: Remove unused member variables:
encoder_bitrate_limits_
quality_scaling_experiment_enabled_

Bug: none
Change-Id: Ifb2b839c826f3d1e416db877d3133ac6ad969000
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/213141
Commit-Queue: Åsa Persson <asapersson@webrtc.org>
Reviewed-by: Sergey Silkin <ssilkin@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#33587}
2021-03-29 14:58:05 +00:00
73cf80a932 Fixes incorrect feedback to EncoderBitrateAdjuster [perf note]
At the point where an EncodedImage is reported to the
EncoderBitrateAdjuster (in order to estimate utilization), the image
data has been cleared so the size is 0 - meaning the esimtated
utilization is 0 so pushback is in effect only applied at the
beginning before an estimate is available.

This CL fixes that by explicitly using spatial/temporal id and size in
bytes, rather than passing along the EncodedImage proxy.

It is unclear when this broke, but the regression seems rather old.

This CL will affect the encoded bitrate (and thus indirectly BWE
ramp-up rate), but should avoid exessive delay at low bitrates.
Perf bots will likely trigger alerts, this is expected.

In case there are undesired side-effects, we can entirely disable the
adjuster using existing field-trials.

Bug: webrtc:12606
Change-Id: I936c2045f554696d8b4bb518eee6871ffc12c47d
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/212900
Commit-Queue: Erik Språng <sprang@webrtc.org>
Reviewed-by: Philip Eliasson <philipel@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#33550}
2021-03-24 12:08:23 +00:00
56db9ff1e1 VideoStreamEncoder: Don't map kNative video frame buffers.
Follow-up CL to VP8 and VP9 encoders taking care of mapping.
Context again:
  This CL is part of Optimized Scaling efforts. In Chromium, the native
frame buffer is getting an optimized CropAndScale() implementation. To
support HW accelerated scaling, returning pre-scaled images and skipping
unnecessary intermediate downscales, WebRTC needs to 1) use CropAndScale
instead of libyuv::XXXXScale and 2) only map buffers it actually intends
to encode.

In this CL, VideoStreamEncoder no longer calls GetMappedFrameBuffer() on
behalf of the encoders, since the encoders are now able to either do the
mapping or performs ToI420() anyway.

- Tests for old VSE behaviors are updated to test the new behavior (i.e.
  that native frames are pretty much always forwarded).
- The "having to call ToI420() twice" workaround to Android bug
  https://crbug.com/webrtc/12602 is added to H264 and AV1 encoders.

Bug: webrtc:12469
Change-Id: Ibdc2e138d4782a140f433c8330950e61b9829f43
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/211940
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Evan Shrubsole <eshr@google.com>
Cr-Commit-Position: refs/heads/master@{#33548}
2021-03-24 09:43:11 +00:00
0e42cf703b Reland "Parse encoded frame QP if not provided by encoder"
This reverts commit 727d2afc4330efebc904e0e4f366e885d7b08787.

Reason for revert: Use thread-safe wrapper for H264 parser.

Original change's description:
> Revert "Parse encoded frame QP if not provided by encoder"
>
> This reverts commit 8639673f0c098efc294a7593fa3bd98e28ab7508.
>
> Reason for revert: linux_tsan fails https://ci.chromium.org/ui/p/webrtc/builders/ci/Linux%20Tsan%20v2/25329/overview
>
> Original change's description:
> > Parse encoded frame QP if not provided by encoder
> >
> > Bug: webrtc:12542
> > Change-Id: Ic70b46e226f158db7a478a9f20e1f940804febba
> > Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/210966
> > Commit-Queue: Sergey Silkin <ssilkin@webrtc.org>
> > Reviewed-by: Åsa Persson <asapersson@webrtc.org>
> > Cr-Commit-Position: refs/heads/master@{#33434}
>
> TBR=asapersson@webrtc.org,ssilkin@webrtc.org
>
> Change-Id: Ie251d8f70f8e87fd86b63730aefd2ef3f941e4bb
> No-Presubmit: true
> No-Tree-Checks: true
> No-Try: true
> Bug: webrtc:12542
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/211355
> Reviewed-by: Sergey Silkin <ssilkin@webrtc.org>
> Commit-Queue: Sergey Silkin <ssilkin@webrtc.org>
> Cr-Commit-Position: refs/heads/master@{#33441}

# Not skipping CQ checks because this is a reland.

Bug: webrtc:12542
Change-Id: Ib7601fd6f2f26bceddbea2b4ba54d67a281f3a59
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/211660
Reviewed-by: Sergey Silkin <ssilkin@webrtc.org>
Reviewed-by: Åsa Persson <asapersson@webrtc.org>
Commit-Queue: Sergey Silkin <ssilkin@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#33458}
2021-03-15 10:11:22 +00:00
fd87944042 Removed WebRTC-NetworkCondition-EncoderSwitch field trial.
Bug: webrtc:12474
Change-Id: I50b3219c0dc9d8a63ff097ee6a71c04fe903aea9
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/211663
Reviewed-by: Åsa Persson <asapersson@webrtc.org>
Commit-Queue: Philip Eliasson <philipel@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#33449}
2021-03-12 16:12:55 +00:00
727d2afc43 Revert "Parse encoded frame QP if not provided by encoder"
This reverts commit 8639673f0c098efc294a7593fa3bd98e28ab7508.

Reason for revert: linux_tsan fails https://ci.chromium.org/ui/p/webrtc/builders/ci/Linux%20Tsan%20v2/25329/overview 

Original change's description:
> Parse encoded frame QP if not provided by encoder
>
> Bug: webrtc:12542
> Change-Id: Ic70b46e226f158db7a478a9f20e1f940804febba
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/210966
> Commit-Queue: Sergey Silkin <ssilkin@webrtc.org>
> Reviewed-by: Åsa Persson <asapersson@webrtc.org>
> Cr-Commit-Position: refs/heads/master@{#33434}

TBR=asapersson@webrtc.org,ssilkin@webrtc.org

Change-Id: Ie251d8f70f8e87fd86b63730aefd2ef3f941e4bb
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: webrtc:12542
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/211355
Reviewed-by: Sergey Silkin <ssilkin@webrtc.org>
Commit-Queue: Sergey Silkin <ssilkin@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#33441}
2021-03-11 17:06:06 +00:00
8639673f0c Parse encoded frame QP if not provided by encoder
Bug: webrtc:12542
Change-Id: Ic70b46e226f158db7a478a9f20e1f940804febba
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/210966
Commit-Queue: Sergey Silkin <ssilkin@webrtc.org>
Reviewed-by: Åsa Persson <asapersson@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#33434}
2021-03-11 11:48:38 +00:00
4a4273bf05 VP9 ResolutionBitrateLimits: If bitrates are configured, use intersection.
Bug: none
Change-Id: I58ada41d7a196837b35df4cf61f7e7561998cf13
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/209704
Reviewed-by: Sergey Silkin <ssilkin@webrtc.org>
Commit-Queue: Åsa Persson <asapersson@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#33390}
2021-03-05 17:39:04 +00:00
a86b29be01 Add VP9-specific default resolution bitrate limits
Bug: none
Change-Id: Ifb6f962f04b1f05d20f80a721b1f41904e0a7e99
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/209702
Reviewed-by: Åsa Persson <asapersson@webrtc.org>
Reviewed-by: Artem Titov <titovartem@webrtc.org>
Commit-Queue: Sergey Silkin <ssilkin@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#33389}
2021-03-05 14:28:14 +00:00
258e9899f4 Use default ResolutionBitrateLimits for simulcast with one active stream if not configured
Bug: none
Change-Id: I049dd0924adc43ce249a8eda63cdcb13da42b030
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/208541
Reviewed-by: Sergey Silkin <ssilkin@webrtc.org>
Commit-Queue: Åsa Persson <asapersson@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#33343}
2021-02-25 12:14:45 +00:00
1124ed1ab2 Communicate encoder resolutions via rtc::VideoSinkWants.
This will allow us to optimize the internal buffers of
webrtc::VideoFrame for the resolution(s) that we actually want to
encode.

Bug: webrtc:12469, chromium:1157072
Change-Id: If378b52b5e35aa9a9800c1f7dfe189437ce43253
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/208540
Reviewed-by: Niels Moller <nisse@webrtc.org>
Reviewed-by: Harald Alvestrand <hta@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#33342}
2021-02-25 11:10:55 +00:00
1dd94a023a Use pixels from single active stream if set in CanDecreaseResolutionTo
Simulcast with one active stream:
Use pixels from single active stream if set (instead of input stream which could be larger) to avoid going below the min_pixel_per_frame limit when downgrading resolution.

Bug: none
Change-Id: I65acb12cc53e46f726ccb5bfab8ce08ff0c4cf78
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/208101
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Commit-Queue: Åsa Persson <asapersson@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#33309}
2021-02-22 10:25:32 +00:00
d15a575ec3 Use SequenceChecker from public API
Bug: webrtc:12419
Change-Id: I00cca16a0ec70246156ba00b97aa7ae5ccbf5364
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/205323
Commit-Queue: Artem Titov <titovartem@webrtc.org>
Reviewed-by: Tommi <tommi@webrtc.org>
Reviewed-by: Danil Chapovalov <danilchap@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#33220}
2021-02-10 15:04:55 +00:00
7f354f8606 Use bandwidth allocation in DropDueToSize when incoming resolution increases.
Use bandwidth allocation instead of encoder target bitrate in DropDueToSize when incoming resolution increases to avoid downgrades due to target bitrate being limited by the max bitrate at low resolutions.

Bug: none
Change-Id: Ic41b31c1a86911d4e97b61b0cbc41ce0da739bd4
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/205622
Commit-Queue: Åsa Persson <asapersson@webrtc.org>
Reviewed-by: Sergey Silkin <ssilkin@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#33168}
2021-02-04 17:26:21 +00:00
c91c4233e3 LibvpxVp9Encoder: add option to configure resolution_bitrate_limits.
Bug: none
Change-Id: Icdd7333296d652b1e0c159226df702084303475c
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/204701
Reviewed-by: Sergey Silkin <ssilkin@webrtc.org>
Commit-Queue: Åsa Persson <asapersson@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#33121}
2021-02-01 11:48:00 +00:00
1aa1d6463a Ensure VideoLayersAllocation.frame_rate_fps can not overflow
Bug: webrtc:12000
Change-Id: I14d5f0f987fb20bd74e0428b3791bf370476296e
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/204220
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Per Kjellander <perkj@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#33089}
2021-01-28 12:39:30 +00:00
461b1d903f Restart CPU overuse detection when encoder settings has changed.
This could potentially lead to unnecessary restarts since it is also
started after the encoder is created. However, it is needed since the
hardware acceleration support can change even though the encoder has
not been recreated.

Bug: b/145730598
Change-Id: Iad1330e7c7bdf769a68c4ecf7abb6abbf3a4fe71
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/203140
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Jakob Ivarsson <jakobi@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#33060}
2021-01-22 17:10:12 +00:00
a7e34d33fe Add resolution_bitrate_limits to EncoderInfo field trial.
Added class EncoderInfoSettings for parsing settings.
Added use of class to SimulcastEncoderAdapter.

Bug: none
Change-Id: I8182b2ab43f0c330ebdf077e9f7cbc79247da90e
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/202246
Commit-Queue: Åsa Persson <asapersson@webrtc.org>
Reviewed-by: Sergey Silkin <ssilkin@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#33050}
2021-01-21 07:53:57 +00:00
a24d35eb09 AlignmentAdjuster: take reduced layers into account for default downscaling.
Bug: none
Change-Id: Id70f7763d2e1b11c24ad98774f1bf6a661728437
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/202257
Reviewed-by: Rasmus Brandt <brandtr@webrtc.org>
Commit-Queue: Åsa Persson <asapersson@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#33038}
2021-01-19 16:59:11 +00:00
b03b6c8a94 Move setting of encoder bitrate allocation callback type to VideoSendStream
It turned out that the negotiated rtp header extensions are not fully known in WebRtcVideoChannel::AddSendStream.

The cl also remove the unnecessary factory for creating VideoStreamEncoder.


Bug: webrtc:12000
Change-Id: If994c8deb69f3ce4212896d3ad757dac94c6e09f
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/198840
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Per Kjellander <perkj@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#32916}
2021-01-07 09:29:05 +00:00
f86cf4c2de Add support for VideoLayersAllocation for Vp9 scv/ksvc and none scalable
VideoCodecInitializer::VideoEncoderConfigToVideoCodec is modified to always set correct frame rate, width and height on spatial layer 0 so the rest of the code does not need to differentiate between scalable/none scalable codecs.


Bug: webrtc:12000
Change-Id: I5a068b98ca2038621205f55e4024f949ab51587a
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/198540
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Per Kjellander <perkj@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#32890}
2020-12-30 16:45:03 +00:00
ad70609509 Implement fake PixelLimitResource for TestBed.
This CL implements a Resource that aggressively reports overuse or
underuse until the encoded stream has the max pixels specified. The
pixel limit is controlled with a field trial, e.g:

--force-fieldtrials="WebRTC-PixelLimitResource/Enabled-307200/"

This caps the resolution to 307200 (=640x480). This can be used by the
TestBed to simulate being CPU limited. Note that the resource doesn't
care about degradation preference at the moment, so if the degradation
preference would be set to "maintain-resolution" the PixelLimitResource
would never stop reporting overuse and we would quickly get a low-FPS
stream.

PixelLimitResource runs a repeating task and reports overuse, underuse
or neither every 5 seconds. This ensures we quickly reach the desired
resolution.

Unit tests are added. I did not add any integration tests (I think
that's overkill for a testing-only resource) but I have manually
verified that this works as intended.

This CL also moves the FakeVideoStreamInputStateProvider into a test/
folder and exposes video_stream_adapter.cc's GetLowerResolutionThan().

Bug: webrtc:12261
Change-Id: Ifbf7c4c05e9dd2843543589bebef3f49b18c38c0
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/195600
Reviewed-by: Evan Shrubsole <eshr@google.com>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Commit-Queue: Henrik Boström <hbos@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#32771}
2020-12-04 10:35:53 +00:00
cde4a9f669 Enable initial frame drop for SVC 'singlecast'
Bug: none
Change-Id: Ideda726f4f7df5e92556048a199cda06261e76b4
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/195542
Reviewed-by: Åsa Persson <asapersson@webrtc.org>
Commit-Queue: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#32714}
2020-11-27 14:08:45 +00:00
20e4c80fbe Reland "Introduce RTC_NO_UNIQUE_ADDRESS."
This is a reland of f5e261aaf65cdf2eb903cdf40d651846be44f447

This CL disables RTC_NO_UNIQUE_ADDRESS on MSan builds since
there have been some issues.

Original change's description:
> Introduce RTC_NO_UNIQUE_ADDRESS.
>
> This macro introduces the possibility to suggest the compiler that a
> data member doesn't need an address different from other non static
> data members.
>
> The usage of a macro is to maintain portability since at the moment
> the attribute [[no_unique_address]] is only supported by clang
> with at least -std=c++11 but it should be supported by all the
> compilers starting from C++20.
>
> Bug: webrtc:11495
> Change-Id: I9f12b67b4422a2749649eaa6b004a67d5fd572d8
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/173331
> Commit-Queue: Mirko Bonadei <mbonadei@webrtc.org>
> Reviewed-by: Karl Wiberg <kwiberg@webrtc.org>
> Cr-Commit-Position: refs/heads/master@{#32246}

Bug: webrtc:11495, webrtc:12218
Change-Id: I4e6c7cc37d3daffad2407c9a2acfa897fa5b426a
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/189968
Commit-Queue: Mirko Bonadei <mbonadei@webrtc.org>
Reviewed-by: Karl Wiberg <kwiberg@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#32668}
2020-11-23 11:29:36 +00:00
f46723c8aa Enable initial frame drop for one active simulcast layer.
Bug: webrtc:12216
Change-Id: Ib2ac2fab45e560ba3eae30a926ce72667a257b07
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/193840
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Commit-Queue: Åsa Persson <asapersson@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#32652}
2020-11-20 16:21:05 +00:00
87e99095a7 Make video scalability mode configurable from peerconnection level.
This CL does not aim at cleaning up simulcast/SVC configuration, just to make it possible to set the scalability mode for AV1. Implementing a codec agnostic SVC/simulcast API is a (big) project on its own.

Change-Id: Ia88df31eb1111713e5f8832e95c8db44f92887ca

BUG: webrtc:11607
Change-Id: Ia88df31eb1111713e5f8832e95c8db44f92887ca
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/192541
Reviewed-by: Florent Castelli <orphis@webrtc.org>
Reviewed-by: Christoffer Rodbro <crodbro@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Philip Eliasson <philipel@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#32631}
2020-11-18 12:06:03 +00:00
c4ad6062fd Ensure EncoderSink::OnBitrateAllocationUpdated not called after Stop
Bug: chromium:1143311, webrtc:12000
Change-Id: I149e960a4999442b289f4b3c576206cc6baf6f24
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/193063
Reviewed-by: Erik Språng <sprang@webrtc.org>
Reviewed-by: Per Kjellander <perkj@webrtc.org>
Commit-Queue: Per Kjellander <perkj@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#32605}
2020-11-13 16:26:41 +00:00
9d69cbeabf Changes default pacing factor to 1.1x
This changes the default behavior to use pacing factor of 1.1x instead
of 2.5x, it also sets libvpx rate controler as trusted, turns on the
encoder pushback mechanism and sets spatial hysteresis to 1.2.
The unused "dynamic rate" settings in libvpx is removed.

The new settings matches what has been used in chromium since 2019.
If needed, the legacy behavior can be enabled using the field trial
WebRTC-VideoRateControl.

Bug: webrtc:10155
Change-Id: I8186b491aa5bef61e8f568e96c980ca68f0c208f
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/186661
Commit-Queue: Erik Språng <sprang@webrtc.org>
Reviewed-by: Per Kjellander <perkj@webrtc.org>
Reviewed-by: Christoffer Rodbro <crodbro@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#32477}
2020-10-23 13:43:32 +00:00
a94348440b VideoStreamEncoder report VideoLayersAllocation for simulcast
Adds support for Vp8 simulcast.

Bug: webrtc:12000
Change-Id: Ib24fd0542642b023ec35f7a7bdc4880d72365edf
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/187341
Commit-Queue: Per Kjellander <perkj@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#32416}
2020-10-15 17:39:06 +00:00
9577be3693 Use VideoFrameBuffer::CropAndScale in VideoStreamEncoder
This avoids a conversion to I420 for frames that support crop
and scale.

Bug: webrtc:11976
Change-Id: I6517a016403cff3ea7ebce1f3de9f9af8b569933
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/187357
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Commit-Queue: Evan Shrubsole <eshr@google.com>
Cr-Commit-Position: refs/heads/master@{#32366}
2020-10-09 13:02:46 +00:00
b556b08668 Allow encoders to receive preferred pixel formats from native buffers
Adds a field to EncoderInfo called preferred_pixel_formats which a
software encoder populates with the pixel formats it supports. When a
kNative frame is received for encoding, the VideoStreamEncoder will
first try to get a frame that is accessible by the software encoder in
that pixel format from the kNative frame. If this fails it will fallback
to converting the frame using ToI420.

This minimizes the number of conversions made in the case that the
encoder supports the pixel format of the native buffer or where
conversion can be accelerated. For example, in Chromium, the capturer can
emit an NV12 frame, which can be consumed by libvpx which supports NV12.

Testing: Tested in Chrome with media::VideoFrame adapters.

Bug: webrtc:11977
Change-Id: I9becc4100136b0c0128f4fa06dedf9ee4dc62f37
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/187121
Reviewed-by: Niels Moller <nisse@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Markus Handell <handellm@webrtc.org>
Commit-Queue: Evan Shrubsole <eshr@google.com>
Cr-Commit-Position: refs/heads/master@{#32353}
2020-10-08 13:47:50 +00:00
dcef6410b3 Stop using VideoBitrateAllocationObserver in VideoStreamEncoder.
VideoBitrateAllocation is instead reported through the EncoderSink.
Enable VideoBitrateAllocation reporting from WebRtcVideoChannel::AddSendStream in preparation for
using the extension RtpVideoLayersAllocationExtension instead of RTCP XR.

Bug: webrtc:12000
Change-Id: I5ea8e4f237a1c4e84a89cbfd97ac4353d4c2984f
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/186940
Commit-Queue: Per Kjellander <perkj@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#32347}
2020-10-07 18:01:13 +00:00
d0a8f51ef7 Reland Refactor reporting of VideoBitrateAllocation
Original description
 Move reporting of target bitrate to just after the encoder has been
 updated. Originall submitted as refs/heads/master@{#32275}

Patch 1 contains the original cl
,patch 2 the fix to send rtcp even if BWE does not change.

Bug: webrtc:12000
Change-Id: I16766e08229fe1f6f65f449e0e074bed03338693
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/186948
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Per Kjellander <perkj@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#32340}
2020-10-07 11:06:56 +00:00
b8781796d4 Revert "Refactor reporting of VideoBitrateAllocation"
This reverts commit 39a31afb77e3ce5c4ff53b8bab06364712cae7ce.

Reason for revert: Will cause RTCP Target bitrate messages to not be sent unless BWE changes. 

Original change's description:
> Refactor reporting of VideoBitrateAllocation
>
> Move reporting of target bitrate to just after the encoder has been
> updated.
>
> Bug: webrtc:12000
> Change-Id: I3e7c5bd44c2f64e5f7e32d6451861b80e0b779ca
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/186041
> Commit-Queue: Per Kjellander <perkj@webrtc.org>
> Reviewed-by: Erik Språng <sprang@webrtc.org>
> Cr-Commit-Position: refs/heads/master@{#32275}

TBR=sprang@webrtc.org,perkj@webrtc.org

# Not skipping CQ checks because original CL landed > 1 day ago.

Bug: webrtc:12000
Change-Id: Icf21e6ae28dc17c61b9243c037ffef9b623894ee
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/186945
Reviewed-by: Per Kjellander <perkj@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Per Kjellander <perkj@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#32337}
2020-10-07 08:14:21 +00:00
0abd518abd Revert "Introduce RTC_NO_UNIQUE_ADDRESS."
This reverts commit f5e261aaf65cdf2eb903cdf40d651846be44f447.

Reason for revert: Breaks downstream projects.

Original change's description:
> Introduce RTC_NO_UNIQUE_ADDRESS.
>
> This macro introduces the possibility to suggest the compiler that a
> data member doesn't need an address different from other non static
> data members.
>
> The usage of a macro is to maintain portability since at the moment
> the attribute [[no_unique_address]] is only supported by clang
> with at least -std=c++11 but it should be supported by all the
> compilers starting from C++20.
>
> Bug: webrtc:11495
> Change-Id: I9f12b67b4422a2749649eaa6b004a67d5fd572d8
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/173331
> Commit-Queue: Mirko Bonadei <mbonadei@webrtc.org>
> Reviewed-by: Karl Wiberg <kwiberg@webrtc.org>
> Cr-Commit-Position: refs/heads/master@{#32246}

TBR=mbonadei@webrtc.org,kwiberg@webrtc.org

# Not skipping CQ checks because original CL landed > 1 day ago.

Bug: webrtc:11495
Change-Id: Ice318d1b11ca3dff09c190187a0b0a32ca945fe3
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/186944
Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org>
Reviewed-by: Per Kjellander <perkj@webrtc.org>
Commit-Queue: Mirko Bonadei <mbonadei@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#32335}
2020-10-07 07:37:01 +00:00
895556e19c Avoid converting frames to I420 in VideoStreamEncoder
This needs to be done still for kNative frames, but all other frame types
can be passed in.

I have checked all VideoEncoder implementations in Chromium and confirmed they either convert the frame to their preferred pixel format, or just
forward the frame to a delegate encoder.

Tested:
- video_loopback with NV12 generated frames for VP9, the only
codec supporting NV12, as well as VP8 which only accepts I420 frames.
- internal_tests tryrun

Bug: webrtc:11976,webrtc:11635
Change-Id: If39a815fb0c5636fceb1040c8946c3db2fb350a1
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/185803
Commit-Queue: Evan Shrubsole <eshr@google.com>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Niels Moller <nisse@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#32306}
2020-10-05 08:33:46 +00:00
39a31afb77 Refactor reporting of VideoBitrateAllocation
Move reporting of target bitrate to just after the encoder has been
updated.

Bug: webrtc:12000
Change-Id: I3e7c5bd44c2f64e5f7e32d6451861b80e0b779ca
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/186041
Commit-Queue: Per Kjellander <perkj@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#32275}
2020-10-01 14:10:09 +00:00
f5e261aaf6 Introduce RTC_NO_UNIQUE_ADDRESS.
This macro introduces the possibility to suggest the compiler that a
data member doesn't need an address different from other non static
data members.

The usage of a macro is to maintain portability since at the moment
the attribute [[no_unique_address]] is only supported by clang
with at least -std=c++11 but it should be supported by all the
compilers starting from C++20.

Bug: webrtc:11495
Change-Id: I9f12b67b4422a2749649eaa6b004a67d5fd572d8
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/173331
Commit-Queue: Mirko Bonadei <mbonadei@webrtc.org>
Reviewed-by: Karl Wiberg <kwiberg@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#32246}
2020-09-30 09:52:49 +00:00
1e4d4fdf88 Don't trigger key frame when encoder is not reset during reconfigure
Currently, key frames are scheduled even when the encoder is not reset
during reconfigeration. This means whenever new parameters like max
bitrate or min bitrate are updated through SetRtpParameters(), the
triggered encoder reconfigeration will always schedule key frames even
they are not necessary. Since parameters' changes like bitrate doesn't
require encoder instance reset.
This causes flood of key frames in our app since we do regularly max
bitrate update according to server control message.


Bug: None
Change-Id: I15d953b24c30e6026c0e97b30f44495d845f293f
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/185380
Commit-Queue: Rasmus Brandt <brandtr@webrtc.org>
Reviewed-by: Rasmus Brandt <brandtr@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#32245}
2020-09-30 08:53:15 +00:00
d41c2a6b8a Remove AsyncInvoker from WebRtcVideoChannel.
RequestEncoderFallback, RequestEncoderSwitch and
SetVideoCodecSwitchingEnabledRequest are now all called on the
worker thread. Before, the work already happened on that thread but
WebRtcVideoChannel adapted internally when needed.

With this CL, there are thread checks to make sure that these calls are
always made the same way, we don't need the async invoker and there
are fewer calls out from the encoder thread in VideoStreamEncoder
(reducing the chance of unintentional blocking).

Bug: webrtc:11908
Change-Id: If8738bc2a708a0fefc6fe850b32655f049f30bdc
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/184603
Commit-Queue: Tommi <tommi@webrtc.org>
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#32151}
2020-09-21 15:04:43 +00:00
612445ea60 Remove use of asyncinvoker from WebRtcVideoSendStream.
This turned out to be a bit complicated, mostly
related to the tests, but here's what's changed:

* No AsyncInvoker (and avoid ClearInternal) in
  WebRtcVideoSendStream (WVSS)
* The reason it was there is due to a "design leak" from
  VideoSourceSinkController/VideoStreamEncoder where the former uses
  locks in all methods and is unaware of a threading model. That design
  affected downstream objects, pushed the need for an async hop into
  WVSS and added a lock.
  A suggestion was made to address this in a follow-up change, here:
  https://webrtc-review.googlesource.com/c/src/+/165684
* All methods in VideoSourceSinkController are now called on a known
  and checked sequence and this CL removes the lock. This also makes
  checking state consistent (i.e. calling a getter twice in a row on the
  same sequence, will always return the same value, avoiding race with
  other threads).
* Handling of reporting state changes from the encoder queue to the
  VSSC, is done by VideoStreamEncoder.
* VideoSendStreamImpl is still instantiated on the incorrect thread [1]
  but has two initialization steps [2]. The second one already runs on
  the right thread. Addressing that TODO [1] is something we should do
  but it has side effects to consider. For the purposes of this CL
  the steps relating to the encoder (setting the sink pointer) have
  been moved to [2].

[1] https://source.chromium.org/chromium/chromium/src/+/master:third_party/webrtc/video/video_send_stream.cc;l=94
[2] https://source.chromium.org/chromium/chromium/src/+/master:third_party/webrtc/video/video_send_stream.cc;drc=f4a9991cce74a37d006438ec0e366313ed33162e;l=115

Bug: webrtc:11222, webrtc:11908
Change-Id: Ie46d46e3a52bbe225951b4bd580ecb8cc9cad873
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/184508
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Commit-Queue: Tommi <tommi@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#32150}
2020-09-21 13:29:53 +00:00
c5a74ffba4 Add support so requested resolution alignment also apply to scaled layers.
Bug: webrtc:11872
Change-Id: I7f904e2765330ee93270b66b0102ce57f336f9a0
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/181883
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Rasmus Brandt <brandtr@webrtc.org>
Commit-Queue: Åsa Persson <asapersson@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#32146}
2020-09-21 09:23:22 +00:00
99b0f8d26c Reland "[Adaptation] Remove QualityScalerResource when disabled."
This is a reland of ba8abbb630cdd9d05e22c830d0845e920762850d

This can be relanded as the queuing issues that were causing a
crash in the WebRTC roll in Chromium have been resolved. I have
added the Chromium failing targets to the CQ for this commit and
they have succeeded.

Original change's description:
> [Adaptation] Remove QualityScalerResource when disabled.
>
> Bug: webrtc:11843
> Change-Id: I2d3e40356c266f189db0242f3c7590e6d83e4456
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/181369
> Commit-Queue: Evan Shrubsole <eshr@google.com>
> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
> Cr-Commit-Position: refs/heads/master@{#31924}

Bug: webrtc:11843
Change-Id: I228331293060ef996f1dd7f8e18d52b0818f526b
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/182080
Commit-Queue: Evan Shrubsole <eshr@google.com>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#31996}
2020-08-26 06:33:43 +00:00
8572841131 [Adaptation] Remove resource adaptation queue
Resource adaptation needs refactoring for async adaptations. For now
the resource adaptation processor can work on the encoder thread, until
it is refactored to support async adaptation.

Bug: webrtc:11867
Change-Id: I9c46da356db19c0fd52748c999ccb216f2ca923b
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/182040
Commit-Queue: Evan Shrubsole <eshr@google.com>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#31991}
2020-08-25 12:16:20 +00:00
2549f174b5 Remove RTPFragmentationHeader creation and propagation through webrtc
Bug: webrtc:6471
Change-Id: I5cb1e10088aaecb5981888082b87ae9957bbaaef
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/181541
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Danil Chapovalov <danilchap@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#31955}
2020-08-17 16:37:33 +00:00
61b2993aaf Revert "[Adaptation] Remove QualityScalerResource when disabled."
This reverts commit ba8abbb630cdd9d05e22c830d0845e920762850d.

Reason for revert: Suspect of causing Chormium trybots to fail, preventing rolls. Will reland if the revert does not fix it.

Original change's description:
> [Adaptation] Remove QualityScalerResource when disabled.
> 
> Bug: webrtc:11843
> Change-Id: I2d3e40356c266f189db0242f3c7590e6d83e4456
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/181369
> Commit-Queue: Evan Shrubsole <eshr@google.com>
> Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
> Cr-Commit-Position: refs/heads/master@{#31924}

TBR=ilnik@webrtc.org,eshr@google.com

# Not skipping CQ checks because original CL landed > 1 day ago.

Bug: webrtc:11843
Change-Id: Idc3950be209c6edce0dbe72d98c9b4becae0049f
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/181880
Reviewed-by: Guido Urdaneta <guidou@webrtc.org>
Commit-Queue: Guido Urdaneta <guidou@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#31952}
2020-08-17 13:46:30 +00:00
d73d421565 [Adaptation] Move min pixel limit logic out of adaptation processor
This is in preperation for eventual multi-stream and multi-mitigation
adaptation. This logic only applied to a single stream and thus is
better fit in the VideoStreamAdapter.

Bug: webrtc:11754
Change-Id: Icc5c7920038c82b574f4b5f7efbc92698691076f
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/181585
Commit-Queue: Evan Shrubsole <eshr@google.com>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#31944}
2020-08-17 10:33:44 +00:00
ba8abbb630 [Adaptation] Remove QualityScalerResource when disabled.
Bug: webrtc:11843
Change-Id: I2d3e40356c266f189db0242f3c7590e6d83e4456
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/181369
Commit-Queue: Evan Shrubsole <eshr@google.com>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#31924}
2020-08-13 08:42:27 +00:00
552eed534c [Adaptation] Don't toggle EncoderUsageResource on/off
Instead it should be ensured that it is started with the correct config.
This removes confusion regarding a resource state. If the resource
check is stopped then the adaptations for that resource should be
removed, and there is no way to determine that if we have one method for
stop for both reconfigure and shutdown.

Bug: webrtc:11843
Change-Id: I491f2fd1f4f803a4610124c7b0026ad75ab4a9cc
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/181368
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Commit-Queue: Evan Shrubsole <eshr@google.com>
Cr-Commit-Position: refs/heads/master@{#31913}
2020-08-11 15:05:10 +00:00
a1c77f6d0d [Adaptation] Move Balanced MinFpsDiff logic to VideoStreamAdapter
This way can double adapt right away instead of relying
on the qp scaler checking soon into the future.

Bug: webrtc:11830
Change-Id: I8e878168303cf6a4c3edcf3997dd8ac2413a4479
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/181060
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Åsa Persson <asapersson@webrtc.org>
Commit-Queue: Evan Shrubsole <eshr@google.com>
Cr-Commit-Position: refs/heads/master@{#31895}
2020-08-10 15:56:07 +00:00