This brings QualityScaler much more in line with OveruseFrameDetector.
The two classes are conceptually similar, and should be used in the
same way. The biggest changes in this CL are:
- Quality scaling is now only done in ViEEncoder and not in each
encoder implementation separately.
- QualityScaler now checks the average QP asynchronously, instead of
having to be polled on each frame.
- QualityScaler is no longer responsible for actually scaling the frames,
but has a callback to ViEEncoder that it uses to express it's desire
for lower resolution.
BUG=webrtc:6495
Review-Url: https://codereview.webrtc.org/2398963003
Cr-Commit-Position: refs/heads/master@{#15286}
There's no longer any need to make the two arguments have the same
signedness, so we can remove a bunch of superfluous (and sometimes
dangerous) casts.
It turned out I also had to fix the safe_cmp functions to properly handle
enums that are implicitly convertible to integers.
NOPRESUBMIT=true
BUG=webrtc:6645
Review-Url: https://codereview.webrtc.org/2534683002
Cr-Commit-Position: refs/heads/master@{#15281}
Pass the selected cricket::VideoCodec to H264EncoderImpl::H264EncoderImpl. The cricket::VideoCodec contains relevant information for H264 about selected profile and packetization mode.
BUG=chromium:600254,webrtc:6402, webrtc:6337
Review-Url: https://codereview.webrtc.org/2474993002
Cr-Commit-Position: refs/heads/master@{#15270}
Set the |qp_| field in EncodedImage before calling OnEncodedImage.
BUG=webrtc:6541
Review-Url: https://codereview.webrtc.org/2499003002
Cr-Commit-Position: refs/heads/master@{#15199}
This is yet another reland of https://codereview.webrtc.org/2434073003/
including two fixes:
1. SimulcastRateAllocator did not handle the screenshare settings properly for numSimulcastStreams = 1. Additional test case was added for that.
2. In VideoSender, when rate allocation is updated after setting a new VideoCodec config, only update the state of the EncoderParameters, but don't actually run SetRateAllocation on the encoder itself. This caused some problems upstreams.
Please review only the changes after patch set 1.
Original description:
Extract bitrate allocation of spatial/temporal layers out of codec impl.
This CL makes a number of intervowen changes:
* Add BitrateAllocation struct, that contains a codec independent view
of how the target bitrate is distributed over spatial and temporal
layers.
* Adds the BitrateAllocator interface, which takes a bitrate and frame
rate and produces a BitrateAllocation.
* A default (non layered) implementation is added, and
SimulcastRateAllocator is extended to fully handle VP8 allocation.
This includes capturing TemporalLayer instances created by the
encoder.
* ViEEncoder now owns both the bitrate allocator and the temporal layer
factories for VP8. This allows allocation to happen fully outside of
the encoder implementation.
This refactoring will make it possible for ViEEncoder to signal the
full picture of target bitrates to the RTCP module.
BUG=webrtc:6301
R=stefan@webrtc.org
Review URL: https://codereview.webrtc.org/2510583002 .
Cr-Commit-Position: refs/heads/master@{#15105}
This approach extends the H.264 specific information with
a packetization mode enum.
Status: Parameter is in code. No way to set it yet.
Rebase of CL 2009213002
BUG=600254
Review-Url: https://codereview.webrtc.org/2337453002
Cr-Commit-Position: refs/heads/master@{#15032}
Reason for revert:
Seems to be causing flakiness in perf test:
FullStackTest.ScreenshareSlidesVP8_2TL_LossyNet
Original issue's description:
> Reland of Issue 2434073003: Extract bitrate allocation ...
>
> This is a reland of https://codereview.webrtc.org/2434073003/ including
> some fixes for failing test cases.
>
> Original description:
>
> Extract bitrate allocation of spatial/temporal layers out of codec impl.
>
> This CL makes a number of intervowen changes:
>
> * Add BitrateAllocation struct, that contains a codec independent view
> of how the target bitrate is distributed over spatial and temporal
> layers.
>
> * Adds the BitrateAllocator interface, which takes a bitrate and frame
> rate and produces a BitrateAllocation.
>
> * A default (non layered) implementation is added, and
> SimulcastRateAllocator is extended to fully handle VP8 allocation.
> This includes capturing TemporalLayer instances created by the
> encoder.
>
> * ViEEncoder now owns both the bitrate allocator and the temporal layer
> factories for VP8. This allows allocation to happen fully outside of
> the encoder implementation.
>
> This refactoring will make it possible for ViEEncoder to signal the
> full picture of target bitrates to the RTCP module.
>
> BUG=webrtc:6301
>
> Committed: https://crrev.com/647bf43dcb2fd16fccf276bd94dc4400728bb405
> Cr-Commit-Position: refs/heads/master@{#15023}
TBR=mflodman@webrtc.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG=webrtc:6301
Review-Url: https://codereview.webrtc.org/2491393002
Cr-Commit-Position: refs/heads/master@{#15026}
This is a reland of https://codereview.webrtc.org/2434073003/ including
some fixes for failing test cases.
Original description:
Extract bitrate allocation of spatial/temporal layers out of codec impl.
This CL makes a number of intervowen changes:
* Add BitrateAllocation struct, that contains a codec independent view
of how the target bitrate is distributed over spatial and temporal
layers.
* Adds the BitrateAllocator interface, which takes a bitrate and frame
rate and produces a BitrateAllocation.
* A default (non layered) implementation is added, and
SimulcastRateAllocator is extended to fully handle VP8 allocation.
This includes capturing TemporalLayer instances created by the
encoder.
* ViEEncoder now owns both the bitrate allocator and the temporal layer
factories for VP8. This allows allocation to happen fully outside of
the encoder implementation.
This refactoring will make it possible for ViEEncoder to signal the
full picture of target bitrates to the RTCP module.
BUG=webrtc:6301
Review-Url: https://codereview.webrtc.org/2488833004
Cr-Commit-Position: refs/heads/master@{#15023}
Reason for revert:
Breaks perf tests.
Original issue's description:
> Extract bitrate allocation of spatial/temporal layers out of codec impl.
>
> This CL makes a number of intervowen changes:
>
> * Add BitrateAllocation struct, that contains a codec independent view
> of how the target bitrate is distributed over spatial and temporal
> layers.
>
> * Adds the BitrateAllocator interface, which takes a bitrate and frame
> rate and produces a BitrateAllocation.
>
> * A default (non layered) implementation is added, and
> SimulcastRateAllocator is extended to fully handle VP8 allocation.
> This includes capturing TemporalLayer instances created by the
> encoder.
>
> * ViEEncoder now owns both the bitrate allocator and the temporal layer
> factories for VP8. This allows allocation to happen fully outside of
> the encoder implementation.
>
> This refactoring will make it possible for ViEEncoder to signal the
> full picture of target bitrates to the RTCP module.
>
> BUG=webrtc:6301
>
> Committed: https://crrev.com/8f46c679d24a05b3f08e02c6d91ec9637f34e24f
> Cr-Commit-Position: refs/heads/master@{#14998}
TBR=stefan@webrtc.org,perkj@webrtc.org,mflodman@webrtc.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG=webrtc:6301
Review-Url: https://codereview.webrtc.org/2489843002
Cr-Commit-Position: refs/heads/master@{#15001}
This CL makes a number of intervowen changes:
* Add BitrateAllocation struct, that contains a codec independent view
of how the target bitrate is distributed over spatial and temporal
layers.
* Adds the BitrateAllocator interface, which takes a bitrate and frame
rate and produces a BitrateAllocation.
* A default (non layered) implementation is added, and
SimulcastRateAllocator is extended to fully handle VP8 allocation.
This includes capturing TemporalLayer instances created by the
encoder.
* ViEEncoder now owns both the bitrate allocator and the temporal layer
factories for VP8. This allows allocation to happen fully outside of
the encoder implementation.
This refactoring will make it possible for ViEEncoder to signal the
full picture of target bitrates to the RTCP module.
BUG=webrtc:6301
Review-Url: https://codereview.webrtc.org/2434073003
Cr-Commit-Position: refs/heads/master@{#14998}
Reason for revert:
Fix gyp build
Original issue's description:
> Revert of Add a webrtc{en,de}coderfactory implementation for VideoToolbox (patchset #2 id:20001 of https://codereview.webrtc.org/2463313002/ )
>
> Reason for revert:
> Broke dependent project because the .gn changes weren't accompanied by corresponding .gyp changes.
>
> Original issue's description:
> > Add a webrtc{en,de}coderfactory implementation for VideoToolbox
> >
> > This CL removes the coupling of the VideoToolbox h264 implementation
> > to the generic h264 code. The files have been moved into sdb/obj/Framework
> > and all dependency on them has been removed from the rest of WebRTC.
> > We now add it as an external encoder via a factory supplied to the
> > CreatePeerConnectionFactory call. This also brings the iOS implementation
> > closer to what we do on Android for MediaCodec.
> >
> > BUG=webrtc:6619
> >
> > Committed: https://crrev.com/6a5047dad31f14f53dd9f8bc1ecde19e1dede2e4
> > Cr-Commit-Position: refs/heads/master@{#14953}
>
> TBR=magjed@webrtc.org,stefan@webrtc.org,kthelgason@webrtc.org
> BUG=webrtc:6619
>
> Committed: https://crrev.com/d69ad84420d9c0e1c11450c352f6c92e7c9583f1
> Cr-Commit-Position: refs/heads/master@{#14985}
R=magjed@webrtc.orgTBR=kwiberg@webrtc.org, magjed@webrtc.org, stefan@webrtc.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG=webrtc:6619
Review URL: https://codereview.webrtc.org/2487723004 .
Cr-Commit-Position: refs/heads/master@{#14992}
Reason for revert:
Broke dependent project because the .gn changes weren't accompanied by corresponding .gyp changes.
Original issue's description:
> Add a webrtc{en,de}coderfactory implementation for VideoToolbox
>
> This CL removes the coupling of the VideoToolbox h264 implementation
> to the generic h264 code. The files have been moved into sdb/obj/Framework
> and all dependency on them has been removed from the rest of WebRTC.
> We now add it as an external encoder via a factory supplied to the
> CreatePeerConnectionFactory call. This also brings the iOS implementation
> closer to what we do on Android for MediaCodec.
>
> BUG=webrtc:6619
>
> Committed: https://crrev.com/6a5047dad31f14f53dd9f8bc1ecde19e1dede2e4
> Cr-Commit-Position: refs/heads/master@{#14953}
TBR=magjed@webrtc.org,stefan@webrtc.org,kthelgason@webrtc.org
BUG=webrtc:6619
Review-Url: https://codereview.webrtc.org/2483273002
Cr-Commit-Position: refs/heads/master@{#14985}
This CL removes the coupling of the VideoToolbox h264 implementation
to the generic h264 code. The files have been moved into sdb/obj/Framework
and all dependency on them has been removed from the rest of WebRTC.
We now add it as an external encoder via a factory supplied to the
CreatePeerConnectionFactory call. This also brings the iOS implementation
closer to what we do on Android for MediaCodec.
BUG=webrtc:6619
Review-Url: https://codereview.webrtc.org/2463313002
Cr-Commit-Position: refs/heads/master@{#14953}
Removed EncodedImageCallback::Encoded() and RtpRtcp::SendOutgoingData().
These methods should no longer be used anywhere and it's safe to remove
them.
BUG=chromium:621691
Committed: https://crrev.com/c681250aaa2025836db7669694e323898e5c2ca7
Review-Url: https://codereview.webrtc.org/2405173006
Cr-Original-Commit-Position: refs/heads/master@{#14923}
Cr-Commit-Position: refs/heads/master@{#14935}
Reason for revert:
Still breaks internal downstream project.
Sergey: Please update internal project before relanding this.
Original issue's description:
> Remove deprected functions from EncodedImageCallback and RtpRtcp
>
> Removed EncodedImageCallback::Encoded() and RtpRtcp::SendOutgoingData().
> These methods should no longer be used anywhere and it's safe to remove
> them.
>
> BUG=chromium:621691
>
> Committed: https://crrev.com/c681250aaa2025836db7669694e323898e5c2ca7
> Cr-Commit-Position: refs/heads/master@{#14923}
TBR=mflodman@webrtc.org,stefan@webrtc.org,sergeyu@chromium.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG=chromium:621691
Review-Url: https://codereview.webrtc.org/2479643002
Cr-Commit-Position: refs/heads/master@{#14925}
Removed EncodedImageCallback::Encoded() and RtpRtcp::SendOutgoingData().
These methods should no longer be used anywhere and it's safe to remove
them.
BUG=chromium:621691
Review-Url: https://codereview.webrtc.org/2405173006
Cr-Commit-Position: refs/heads/master@{#14923}
Reason for revert:
Breaks everything
Original issue's description:
> Revert of Remove deprected functions from EncodedImageCallback and RtpRtcp (patchset #4 id:100001 of https://codereview.webrtc.org/2405173006/ )
>
> Reason for revert:
> This might be breaking projects downstream.
>
> Original issue's description:
> > Remove deprected functions from EncodedImageCallback and RtpRtcp
> >
> > Removed EncodedImageCallback::Encoded() and RtpRtcp::SendOutgoingData().
> > These methods should no longer be used anywhere and it's safe to remove
> > them.
> >
> > BUG=chromium:621691
> >
> > Committed: https://crrev.com/fa565842718ad178a7562721b25d916fbabc2b92
> > Cr-Commit-Position: refs/heads/master@{#14902}
>
> TBR=mflodman@webrtc.org,stefan@webrtc.org,sergeyu@chromium.org
> # Skipping CQ checks because original CL landed less than 1 days ago.
> NOPRESUBMIT=true
> NOTREECHECKS=true
> NOTRY=true
> BUG=chromium:621691
>
> Committed: https://crrev.com/6c78307a21252c2dbd704f6d5e92a220fb722ed4
> Cr-Commit-Position: refs/heads/master@{#14914}
TBR=mflodman@webrtc.org,stefan@webrtc.org,sergeyu@chromium.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG=chromium:621691
Review-Url: https://codereview.webrtc.org/2467373003
Cr-Commit-Position: refs/heads/master@{#14915}
Reason for revert:
This might be breaking projects downstream.
Original issue's description:
> Remove deprected functions from EncodedImageCallback and RtpRtcp
>
> Removed EncodedImageCallback::Encoded() and RtpRtcp::SendOutgoingData().
> These methods should no longer be used anywhere and it's safe to remove
> them.
>
> BUG=chromium:621691
>
> Committed: https://crrev.com/fa565842718ad178a7562721b25d916fbabc2b92
> Cr-Commit-Position: refs/heads/master@{#14902}
TBR=mflodman@webrtc.org,stefan@webrtc.org,sergeyu@chromium.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG=chromium:621691
Review-Url: https://codereview.webrtc.org/2474433008
Cr-Commit-Position: refs/heads/master@{#14914}
Removed EncodedImageCallback::Encoded() and RtpRtcp::SendOutgoingData().
These methods should no longer be used anywhere and it's safe to remove
them.
BUG=chromium:621691
Review-Url: https://codereview.webrtc.org/2405173006
Cr-Commit-Position: refs/heads/master@{#14902}
This CL replaces the use of webrtc::Video as an internal
variable in the H.264 encoder with the specific fields
that are used by this encoder.
In support of refactorings discussed around:
BUG=600254
Committed: https://crrev.com/2549437b5ccf5aae2e6f1a1491c5f505d1859f9c
Review-Url: https://codereview.webrtc.org/2468903003
Cr-Original-Commit-Position: refs/heads/master@{#14887}
Cr-Commit-Position: refs/heads/master@{#14892}
Reason for revert:
Landed the wrong patchset. Nothing broken.
Original issue's description:
> Remove webrtc::Video from H264 encoder internals
>
> This CL replaces the use of webrtc::Video as an internal
> variable in the H.264 encoder with the specific fields
> that are used by this encoder.
>
> In support of refactorings discussed around:
>
> BUG=600254
>
> Committed: https://crrev.com/2549437b5ccf5aae2e6f1a1491c5f505d1859f9c
> Cr-Commit-Position: refs/heads/master@{#14887}
TBR=magjed@webrtc.org,stefan@webrtc.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG=600254
Review-Url: https://codereview.webrtc.org/2472673002
Cr-Commit-Position: refs/heads/master@{#14888}
This CL replaces the use of webrtc::Video as an internal
variable in the H.264 encoder with the specific fields
that are used by this encoder.
In support of refactorings discussed around:
BUG=600254
Review-Url: https://codereview.webrtc.org/2468903003
Cr-Commit-Position: refs/heads/master@{#14887}
Reason for revert:
Internal project has been fixed
Original issue's description:
> Revert of Move bitstream parser to more appropriate directory. (patchset #4 id:60001 of https://codereview.webrtc.org/2370853005/ )
>
> Reason for revert:
> Breaks internal project
>
> Original issue's description:
> > Move current bitstream parser to more appropriate directory.
> >
> > This CL groups together the code that has to do with parsing H264 bitstreams.
> > This code logically belongs together, and having it in the same directory not
> > only simplifies things from a project structure perspective, but also makes it
> > easier to refactor out common parts incrementally.
> > An added benefit is that this simplifies modular compilation, where for example
> > one would like a build of WebRTC without the H264 codec-specific parts.
> >
> > BUG=webrtc:6338
> >
> > Committed: https://crrev.com/cc6817e9ce4a5ffc73efb660cf0368afbc7d9a4f
> > Cr-Commit-Position: refs/heads/master@{#14684}
>
> TBR=magjed@webrtc.org,stefan@webrtc.org
> # Skipping CQ checks because original CL landed less than 1 days ago.
> NOPRESUBMIT=true
> NOTREECHECKS=true
> NOTRY=true
> BUG=webrtc:6338
>
> Committed: https://crrev.com/f04f14e772f803de39f8a6128e5157127cd35103
> Cr-Commit-Position: refs/heads/master@{#14685}
TBR=magjed@webrtc.org,stefan@webrtc.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG=webrtc:6338
Review-Url: https://codereview.webrtc.org/2434043002
Cr-Commit-Position: refs/heads/master@{#14783}
The API has changed for the slice config of SSpatialLayerConfig as of
OpenH264 v1.6. Update H264EncoderImpl with an ifdef that uses the
correct API depending on what version of OpenH264 is being used.
BUG=webrtc:6583
Review-Url: https://codereview.webrtc.org/2440113002
Cr-Commit-Position: refs/heads/master@{#14762}
This CL makes scaling and cropping lazy in AVFoundationVideoCapturer and
provides optimized paths for SW and HW encoding. For SW encoding, an
efficient NV12 -> I420 cropping and scaling is implemented in
CoreVideoFrameBuffer::NativeToI420. For HW encoding, an efficient NV12 ->
NV12 cropping and scaling is implemented in
CoreVideoFrameBuffer::CropAndScaleTo. The performance improvement over
the existing cropping and scaling is that it is now done in one step
instead of making an intermediary copy of the Y plane.
There might still be room for improvement in the HW path using some HW
support. That will be explored in a future CL.
BUG=b/30939444
Review-Url: https://codereview.webrtc.org/2394483005
Cr-Commit-Position: refs/heads/master@{#14701}
Reason for revert:
Breaks internal project
Original issue's description:
> Move current bitstream parser to more appropriate directory.
>
> This CL groups together the code that has to do with parsing H264 bitstreams.
> This code logically belongs together, and having it in the same directory not
> only simplifies things from a project structure perspective, but also makes it
> easier to refactor out common parts incrementally.
> An added benefit is that this simplifies modular compilation, where for example
> one would like a build of WebRTC without the H264 codec-specific parts.
>
> BUG=webrtc:6338
>
> Committed: https://crrev.com/cc6817e9ce4a5ffc73efb660cf0368afbc7d9a4f
> Cr-Commit-Position: refs/heads/master@{#14684}
TBR=magjed@webrtc.org,stefan@webrtc.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG=webrtc:6338
Review-Url: https://codereview.webrtc.org/2430353004
Cr-Commit-Position: refs/heads/master@{#14685}
This CL groups together the code that has to do with parsing H264 bitstreams.
This code logically belongs together, and having it in the same directory not
only simplifies things from a project structure perspective, but also makes it
easier to refactor out common parts incrementally.
An added benefit is that this simplifies modular compilation, where for example
one would like a build of WebRTC without the H264 codec-specific parts.
BUG=webrtc:6338
Review-Url: https://codereview.webrtc.org/2370853005
Cr-Commit-Position: refs/heads/master@{#14684}
Also provide a new set of thresholds for the VideoToolbox encoder. The new thresholds were experimentally determined to work well on the iPhone 6S, and also adequately on the iPhone 5S.
BUG=webrtc:5678
Review-Url: https://codereview.webrtc.org/2309743002
Cr-Commit-Position: refs/heads/master@{#14420}
This CL removes the use_objc_h264 flag. This means that the VideoToolbox
H264 encoder and decoder will always be built.
BUG=webrtc:4081
NOTRY=TRUE
Review-Url: https://codereview.webrtc.org/2366443003
Cr-Commit-Position: refs/heads/master@{#14372}
Deleted from the VideoFrameBuffer base class.
BUG=webrtc:5921
Review-Url: https://codereview.webrtc.org/2278883002
Cr-Commit-Position: refs/heads/master@{#14317}
If the input to H264VideoToolBoxEncoder is a native CVPixelBuffer and
the quality scaler requests scaling, we fall back to a slow path where
the buffer is converted from NV12 to I420 on the CPU and then uploaded
to a native CVPixelBuffer again. It turns out this scaling is not needed
and that the H264VideoToolBoxEncoder can handle the scaling internally.
BUG=b/30939444
Review-Url: https://codereview.webrtc.org/2258103003
Cr-Commit-Position: refs/heads/master@{#13835}
H.264 frames may have AUD before SPS. This change detects AUD and try to extract SPS and PPS behind AUD.
BUG=webrtc:6173
Review-Url: https://codereview.webrtc.org/2209143002
Cr-Commit-Position: refs/heads/master@{#13765}
We need to check the pool to make sure that the encoder session is valid. Otherwise, it appears as if the encoder just does not output frames.
NOTRY=True
BUG=
Review-Url: https://codereview.webrtc.org/2226383002
Cr-Commit-Position: refs/heads/master@{#13698}
Scaling causes us to work the CPU too much, which very quickly degrades quality. This causes us to at least behave better on good networks.
NOTRY=True
BUG=
Review-Url: https://codereview.webrtc.org/2205763002
Cr-Commit-Position: refs/heads/master@{#13630}
The iOS H264 video toolbox encoder is currently undershooting the
intended bitrate. This seems to be caused by the data rate limit
property. This CL increases the data rate limit to a set
percentage above the intended bitrate to avoid undershooting. The
AverageBitRate property is still set to the intended bitrate, which
keeps it from overshooting the intended bitrate.
BUG=b/28713684
Review-Url: https://codereview.webrtc.org/2177873003
Cr-Commit-Position: refs/heads/master@{#13526}
Support encoding from CVPixelBuffers directly in H264VideoToolboxEncoder.
If the frame needs to be scaled, it will fall back to the previous slow
path:
CVPixelBuffer -> NV12 -> I420 -> (scale) I420 -> NV12 -> CVPixelBuffer.
BUG=webrtc:4081
Review-Url: https://codereview.webrtc.org/2140573002
Cr-Commit-Position: refs/heads/master@{#13439}
