Commit Graph

51 Commits

Author SHA1 Message Date
1369c83b42 Revert of Issue 2434073003: Extract bitrate allocation ... (patchset #4 id:60001 of https://codereview.webrtc.org/2488833004/ )
Reason for revert:
Seems to be causing flakiness in perf test:
FullStackTest.ScreenshareSlidesVP8_2TL_LossyNet

Original issue's description:
> Reland of Issue 2434073003: Extract bitrate allocation ...
>
> This is a reland of https://codereview.webrtc.org/2434073003/ including
> some fixes for failing test cases.
>
> Original description:
>
> Extract bitrate allocation of spatial/temporal layers out of codec impl.
>
> This CL makes a number of intervowen changes:
>
> * Add BitrateAllocation struct, that contains a codec independent view
>   of how the target bitrate is distributed over spatial and temporal
>   layers.
>
> * Adds the BitrateAllocator interface, which takes a bitrate and frame
>   rate and produces a BitrateAllocation.
>
> * A default (non layered) implementation is added, and
>   SimulcastRateAllocator is extended to fully handle VP8 allocation.
>   This includes capturing TemporalLayer instances created by the
>   encoder.
>
> * ViEEncoder now owns both the bitrate allocator and the temporal layer
>   factories for VP8. This allows allocation to happen fully outside of
>   the encoder implementation.
>
> This refactoring will make it possible for ViEEncoder to signal the
> full picture of target bitrates to the RTCP module.
>
> BUG=webrtc:6301
>
> Committed: https://crrev.com/647bf43dcb2fd16fccf276bd94dc4400728bb405
> Cr-Commit-Position: refs/heads/master@{#15023}

TBR=mflodman@webrtc.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG=webrtc:6301

Review-Url: https://codereview.webrtc.org/2491393002
Cr-Commit-Position: refs/heads/master@{#15026}
2016-11-10 16:30:39 +00:00
647bf43dcb Reland of Issue 2434073003: Extract bitrate allocation ...
This is a reland of https://codereview.webrtc.org/2434073003/ including
some fixes for failing test cases.

Original description:

Extract bitrate allocation of spatial/temporal layers out of codec impl.

This CL makes a number of intervowen changes:

* Add BitrateAllocation struct, that contains a codec independent view
  of how the target bitrate is distributed over spatial and temporal
  layers.

* Adds the BitrateAllocator interface, which takes a bitrate and frame
  rate and produces a BitrateAllocation.

* A default (non layered) implementation is added, and
  SimulcastRateAllocator is extended to fully handle VP8 allocation.
  This includes capturing TemporalLayer instances created by the
  encoder.

* ViEEncoder now owns both the bitrate allocator and the temporal layer
  factories for VP8. This allows allocation to happen fully outside of
  the encoder implementation.

This refactoring will make it possible for ViEEncoder to signal the
full picture of target bitrates to the RTCP module.

BUG=webrtc:6301

Review-Url: https://codereview.webrtc.org/2488833004
Cr-Commit-Position: refs/heads/master@{#15023}
2016-11-10 14:46:28 +00:00
4bc98d4e1b Revert of Extract bitrate allocation of spatial/temporal layers out of codec impl. (patchset #17 id:320001 of https://codereview.webrtc.org/2434073003/ )
Reason for revert:
Breaks perf tests.

Original issue's description:
> Extract bitrate allocation of spatial/temporal layers out of codec impl.
>
> This CL makes a number of intervowen changes:
>
> * Add BitrateAllocation struct, that contains a codec independent view
>   of how the target bitrate is distributed over spatial and temporal
>   layers.
>
> * Adds the BitrateAllocator interface, which takes a bitrate and frame
>   rate and produces a BitrateAllocation.
>
> * A default (non layered) implementation is added, and
>   SimulcastRateAllocator is extended to fully handle VP8 allocation.
>   This includes capturing TemporalLayer instances created by the
>   encoder.
>
> * ViEEncoder now owns both the bitrate allocator and the temporal layer
>   factories for VP8. This allows allocation to happen fully outside of
>   the encoder implementation.
>
> This refactoring will make it possible for ViEEncoder to signal the
> full picture of target bitrates to the RTCP module.
>
> BUG=webrtc:6301
>
> Committed: https://crrev.com/8f46c679d24a05b3f08e02c6d91ec9637f34e24f
> Cr-Commit-Position: refs/heads/master@{#14998}

TBR=stefan@webrtc.org,perkj@webrtc.org,mflodman@webrtc.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG=webrtc:6301

Review-Url: https://codereview.webrtc.org/2489843002
Cr-Commit-Position: refs/heads/master@{#15001}
2016-11-09 14:14:56 +00:00
8f46c679d2 Extract bitrate allocation of spatial/temporal layers out of codec impl.
This CL makes a number of intervowen changes:

* Add BitrateAllocation struct, that contains a codec independent view
  of how the target bitrate is distributed over spatial and temporal
  layers.

* Adds the BitrateAllocator interface, which takes a bitrate and frame
  rate and produces a BitrateAllocation.

* A default (non layered) implementation is added, and
  SimulcastRateAllocator is extended to fully handle VP8 allocation.
  This includes capturing TemporalLayer instances created by the
  encoder.

* ViEEncoder now owns both the bitrate allocator and the temporal layer
  factories for VP8. This allows allocation to happen fully outside of
  the encoder implementation.

This refactoring will make it possible for ViEEncoder to signal the
full picture of target bitrates to the RTCP module.

BUG=webrtc:6301

Review-Url: https://codereview.webrtc.org/2434073003
Cr-Commit-Position: refs/heads/master@{#14998}
2016-11-09 13:09:12 +00:00
fffc1e5578 Add functionality for parsing H264 profile-level-id
The new code is only exercised in tests so far. The H264 profile-level-id
parsing is not complete, but it should be enough for our purposes for
now.

BUG=webrtc:6400,webrtc:6337

Review-Url: https://codereview.webrtc.org/2459633002
Cr-Commit-Position: refs/heads/master@{#14850}
2016-10-31 12:56:03 +00:00
b906172e02 Reland of Move bitstream parser to more appropriate directory. (patchset #1 id:1 of https://codereview.webrtc.org/2430353004/ )
Reason for revert:
Internal project has been fixed

Original issue's description:
> Revert of Move bitstream parser to more appropriate directory. (patchset #4 id:60001 of https://codereview.webrtc.org/2370853005/ )
>
> Reason for revert:
> Breaks internal project
>
> Original issue's description:
> > Move current bitstream parser to more appropriate directory.
> >
> > This CL groups together the code that has to do with parsing H264 bitstreams.
> > This code logically belongs together, and having it in the same directory not
> > only simplifies things from a project structure perspective, but also makes it
> > easier to refactor out common parts incrementally.
> > An added benefit is that this simplifies modular compilation, where for example
> > one would like a build of WebRTC without the H264 codec-specific parts.
> >
> > BUG=webrtc:6338
> >
> > Committed: https://crrev.com/cc6817e9ce4a5ffc73efb660cf0368afbc7d9a4f
> > Cr-Commit-Position: refs/heads/master@{#14684}
>
> TBR=magjed@webrtc.org,stefan@webrtc.org
> # Skipping CQ checks because original CL landed less than 1 days ago.
> NOPRESUBMIT=true
> NOTREECHECKS=true
> NOTRY=true
> BUG=webrtc:6338
>
> Committed: https://crrev.com/f04f14e772f803de39f8a6128e5157127cd35103
> Cr-Commit-Position: refs/heads/master@{#14685}

TBR=magjed@webrtc.org,stefan@webrtc.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG=webrtc:6338

Review-Url: https://codereview.webrtc.org/2434043002
Cr-Commit-Position: refs/heads/master@{#14783}
2016-10-26 09:48:24 +00:00
f04f14e772 Revert of Move bitstream parser to more appropriate directory. (patchset #4 id:60001 of https://codereview.webrtc.org/2370853005/ )
Reason for revert:
Breaks internal project

Original issue's description:
> Move current bitstream parser to more appropriate directory.
>
> This CL groups together the code that has to do with parsing H264 bitstreams.
> This code logically belongs together, and having it in the same directory not
> only simplifies things from a project structure perspective, but also makes it
> easier to refactor out common parts incrementally.
> An added benefit is that this simplifies modular compilation, where for example
> one would like a build of WebRTC without the H264 codec-specific parts.
>
> BUG=webrtc:6338
>
> Committed: https://crrev.com/cc6817e9ce4a5ffc73efb660cf0368afbc7d9a4f
> Cr-Commit-Position: refs/heads/master@{#14684}

TBR=magjed@webrtc.org,stefan@webrtc.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG=webrtc:6338

Review-Url: https://codereview.webrtc.org/2430353004
Cr-Commit-Position: refs/heads/master@{#14685}
2016-10-19 17:34:39 +00:00
cc6817e9ce Move current bitstream parser to more appropriate directory.
This CL groups together the code that has to do with parsing H264 bitstreams.
This code logically belongs together, and having it in the same directory not
only simplifies things from a project structure perspective, but also makes it
easier to refactor out common parts incrementally.
An added benefit is that this simplifies modular compilation, where for example
one would like a build of WebRTC without the H264 codec-specific parts.

BUG=webrtc:6338

Review-Url: https://codereview.webrtc.org/2370853005
Cr-Commit-Position: refs/heads/master@{#14684}
2016-10-19 16:31:15 +00:00
e40a7ee007 GN: Exclude suppressions of Chromium Clang warnings for Chromium builds.
These suppressions are causing GN errors when Chromium targets are depending
directly on WebRTC targets (needed for https://codereview.chromium.org/2413103004)

BUG=webrtc:4256
NOTRY=True

Review-Url: https://codereview.webrtc.org/2408133008
Cr-Commit-Position: refs/heads/master@{#14644}
2016-10-17 06:56:20 +00:00
b62dbbe985 GN: Change rtc_source_set targets --> rtc_static_library
This changes most non-test related rtc_source_set targets to be
rtc_static_library instead. Targets without any .cc files are excluded.
This should bring back the build behavior we used to have with GYP
(i.e. same symbols exported in the libjingle_peerconnection.a file, which
are used by some downstream projects).

After doing an Android build with these changes:
$ nm --defined-only -g -C out/Release/lib.unstripped/libjingle_peerconnection_so.so | grep -i createpeerconnectionf
00077c51 T Java_org_webrtc_PeerConnectionFactory_nativeCreatePeerConnectionFactory
$ nm --defined-only -g -C out/Release/obj/webrtc/api/libjingle_peerconnection.a | grep -i createpeerconnectionf
00000001 T webrtc::CreatePeerConnectionFactory(rtc::Thread*, rtc::Thread*, rtc::Thread*, webrtc::AudioDeviceModule*, cricket::WebRtcVideoEncoderFactory*, cricket::WebRtcVideoDecoderFactory*)
00000001 T webrtc::CreatePeerConnectionFactory()

See https://chromium.googlesource.com/chromium/src/+/master/tools/gn/docs/cookbook.md#Note-on-static-libraries
for more details on this.

NOTICE: This should be further cleaned up in the future, to reduce
binary bloat and unnecessary linking time. Right now it's more
important to restore the desired build output though.

BUG=webrtc:6410, chromium:630755

Review-Url: https://codereview.webrtc.org/2361623004
Cr-Commit-Position: refs/heads/master@{#14364}
2016-09-23 07:38:58 +00:00
3a7f35b1c4 GN: Declare resources for targets.
Declare resources for GN targets so that they can be isolated

NOTRY=True
BUG=chromium:497757

Review-Url: https://codereview.webrtc.org/2340753002
Cr-Commit-Position: refs/heads/master@{#14210}
2016-09-14 12:10:06 +00:00
e9cc686293 GN Templates: Move common_inherited_config to the template.
Remove common_inherited_config from the targets and add it to the
template instead.

BUG=webrtc:6187
NOTRY=True

Review-Url: https://codereview.webrtc.org/2311843002
Cr-Commit-Position: refs/heads/master@{#14069}
2016-09-05 13:10:23 +00:00
7a2ce0b738 GN Templates: Move common_config to the template.
Remove common_config from the targets' config and add
it to the template instead.

BUG=webrtc:6187
NOTRY=True

Review-Url: https://codereview.webrtc.org/2300413002
Cr-Commit-Position: refs/heads/master@{#14063}
2016-09-05 08:35:48 +00:00
38a2132b02 GN: Introduce templates.
Defines the rtc_executable, rtc_source_set, rtc_test and
rtc_static_library templates.

These templates provide no functionality yet, but will enable common
configuration to be introduced, avoiding repetition in every target

Changes summary:
- Prepend rtc_ to test, source_set, executable and static_library targets
- Change "configs -= [" to "suppressed_configs += ["
- Include webrtc/build/webrtc.gni where it wasn't included yet
- Delete import("//testing/test.gni"), since rtc_test makes it unnecessary.

BUG=webrtc:6187
TBR=henrik.lundin@webrtc.org,tommi@webrtc.org
NOTRY=True

Review-Url: https://codereview.webrtc.org/2301053002
Cr-Commit-Position: refs/heads/master@{#14043}
2016-09-02 11:10:41 +00:00
32c4a20df7 GN: Fix resource files for iOS test target
In order to get resource files to be properly packaged into
the .app for a unit test on iOS, the resource files needs
to be listed as sources in a bundle_data target.

BUG=webrtc:5949
NOTRY=True

Review-Url: https://codereview.webrtc.org/2292853002
Cr-Commit-Position: refs/heads/master@{#13968}
2016-08-30 09:53:54 +00:00
28a0ffdd52 GN: Synchronize resources between Android and iOS.
iOS tests packaged into an .app uses the same way of
defining resources (the data attribute). Some iOS
simulator tests are failing due to missing resources, so
let's sync them all.

BUG=webrtc:5949
NOTRY=True

Review-Url: https://codereview.webrtc.org/2277753003
Cr-Commit-Position: refs/heads/master@{#13898}
2016-08-24 14:48:48 +00:00
714dd4e532 GN: Update tests to have the correct shard timeout value on Android.
TBR=mflodman@webrtc.org
NOTRY=True

Review-Url: https://codereview.webrtc.org/2231413002
Cr-Commit-Position: refs/heads/master@{#13741}
2016-08-15 09:29:19 +00:00
4e417b242a Reland of Switch to use SequencedTaskChecker instead of ThreadChecker where needed.
(patchset #1 id:1 of https://codereview.webrtc.org/2149553002/ )"
This reverts commit efd902cb1d9bbd81247a3e168f2080beae761d78.

Originally reviewed in https://codereview.webrtc.org/2149553002

The uptream problem should be fixed by https://codereview.webrtc.org/2145393003/

BUG=webrtc:5687
TBR=tommi@webrtc.org

Review-Url: https://codereview.webrtc.org/2152013002
Cr-Commit-Position: refs/heads/master@{#13483}
2016-07-15 06:36:00 +00:00
efd902cb1d Revert of Switch to use SequencedTaskChecker instead of ThreadChecker where needed. (patchset #1 id:1 of https://codereview.webrtc.org/2149553002/ )
Reason for revert:
Breaks Chrome FYI

https://build.chromium.org/p/chromium.webrtc.fyi/builders/Linux%20Builder/builds/8187/steps/compile/logs/stdio

Original issue's description:
> Switch to use SequencedTaskChecker instead of ThreadChecker where needed.
> This cl is in preparation for https://codereview.webrtc.org/2060403002/ Add task queue to Call.
> In the coming cl the video_sender, and i420_buffer_pool will be used on a task queue and therefore SequencedTaskChecker is needed instead of a ThreadChecker.
>
> BUG=webrtc:5687
>
> Committed: https://crrev.com/ec7cef854d85be2d9fd5b1a4e2c59a67feb662eb
> Cr-Commit-Position: refs/heads/master@{#13474}

TBR=tommi@webrtc.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG=webrtc:5687

Review-Url: https://codereview.webrtc.org/2150843003
Cr-Commit-Position: refs/heads/master@{#13475}
2016-07-14 09:33:25 +00:00
ec7cef854d Switch to use SequencedTaskChecker instead of ThreadChecker where needed.
This cl is in preparation for https://codereview.webrtc.org/2060403002/ Add task queue to Call.
In the coming cl the video_sender, and i420_buffer_pool will be used on a task queue and therefore SequencedTaskChecker is needed instead of a ThreadChecker.

BUG=webrtc:5687

Review-Url: https://codereview.webrtc.org/2149553002
Cr-Commit-Position: refs/heads/master@{#13474}
2016-07-14 08:32:09 +00:00
8e8222d0d2 Revert of Split IncomingVideoStream into two implementations, with smoothing and without. (patchset #4 id:290001 of https://codereview.webrtc.org/2071473002/ )
Reason for revert:
Reverting again.  The perf regression does not seem to be related to dropping frames.

Original issue's description:
> Reland of Split IncomingVideoStream into two implementations, with smoothing and without.
>
> Original issue's description:
>
> Split IncomingVideoStream into two implementations, with smoothing and without.
>
> This CL fixes an issue with the non-smoothing implementation where frames were delivered on the decoder thread.  No-smoothing is now done in a separate class that uses a TaskQueue.  The implementation may drop frames if the renderer doesn't keep up and it doesn't block the decoder thread.
>
> Further work done:
>
> * I added TODOs and documentation for VideoReceiveStream::OnFrame, where we today grab 5 locks.
>
> * I removed the Start/Stop methods from the IncomingVideoStream implementations.  Now, when an instance is created, it should be considered to be "running" and when it is deleted, it's "not running".  This saves on resources and also reduces the amount of locking required and I could remove one critical section altogether.
>
> * I changed the VideoStreamDecoder class to not depend on IncomingVideoStream but rather use the generic rtc::VideoSinkInterface<VideoFrame> interface.  This means that any implementation of that interface can be used and the decoder can be made to  just use the 'renderer' from the config.  Once we do that, we can decouple the IncomingVideoStream implementations from the decoder and VideoReceiveStream implementations and leave it up to the application for how to do smoothing.  The app can choose to use the Incoming* classes or roll its own (which may be preferable since applications often have their own scheduling mechanisms).
>
> * The non-smoothing IncomingVideoStream implementation currently allows only 1 outstanding pending frame.  If we exceed that, the current frame won't be delivered to the renderer and instead we deliver the next one (since when this happens, the renderer is falling behind).
>
> * The lifetime of the VideoStreamDecoder instance is now bound to Start/Stop in VideoReceiveStream and not all of the lifetime of VideoReceiveStream.
>
> * Fixed VideoStreamDecoder to unregister callbacks in the dtor that were registered in the ctor. (this was open to a use-after-free regression)
>
> * Delay and callback pointers are now passed via the ctors to the IncomingVideoStream classes.  The thread primitives in the IncomingVideoStream classes are also constructed/destructed at the same time as the owning object, which allowed me to remove one more lock.
>
> * Removed code in the VideoStreamDecoder that could overwrite the VideoReceiveStream render delay with a fixed value of 10ms on construction.  This wasn't a problem with the previous implementation (it would be now though) but seemed to me like the wrong place to be setting that value.
>
> * Made the render delay value in VideoRenderFrames, const.
>
> BUG=chromium:620232
> TBR=mflodman
>
> Committed: https://crrev.com/e03f8787377bbc03a4e00184bb14b7561b108cbb
> Cr-Commit-Position: refs/heads/master@{#13175}

TBR=mflodman@webrtc.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG=chromium:620232

Review-Url: https://codereview.webrtc.org/2071093002
Cr-Commit-Position: refs/heads/master@{#13176}
2016-06-16 22:44:11 +00:00
e03f878737 Reland of Split IncomingVideoStream into two implementations, with smoothing and without.
Original issue's description:

Split IncomingVideoStream into two implementations, with smoothing and without.

This CL fixes an issue with the non-smoothing implementation where frames were delivered on the decoder thread.  No-smoothing is now done in a separate class that uses a TaskQueue.  The implementation may drop frames if the renderer doesn't keep up and it doesn't block the decoder thread.

Further work done:

* I added TODOs and documentation for VideoReceiveStream::OnFrame, where we today grab 5 locks.

* I removed the Start/Stop methods from the IncomingVideoStream implementations.  Now, when an instance is created, it should be considered to be "running" and when it is deleted, it's "not running".  This saves on resources and also reduces the amount of locking required and I could remove one critical section altogether.

* I changed the VideoStreamDecoder class to not depend on IncomingVideoStream but rather use the generic rtc::VideoSinkInterface<VideoFrame> interface.  This means that any implementation of that interface can be used and the decoder can be made to  just use the 'renderer' from the config.  Once we do that, we can decouple the IncomingVideoStream implementations from the decoder and VideoReceiveStream implementations and leave it up to the application for how to do smoothing.  The app can choose to use the Incoming* classes or roll its own (which may be preferable since applications often have their own scheduling mechanisms).

* The non-smoothing IncomingVideoStream implementation currently allows only 1 outstanding pending frame.  If we exceed that, the current frame won't be delivered to the renderer and instead we deliver the next one (since when this happens, the renderer is falling behind).

* The lifetime of the VideoStreamDecoder instance is now bound to Start/Stop in VideoReceiveStream and not all of the lifetime of VideoReceiveStream.

* Fixed VideoStreamDecoder to unregister callbacks in the dtor that were registered in the ctor. (this was open to a use-after-free regression)

* Delay and callback pointers are now passed via the ctors to the IncomingVideoStream classes.  The thread primitives in the IncomingVideoStream classes are also constructed/destructed at the same time as the owning object, which allowed me to remove one more lock.

* Removed code in the VideoStreamDecoder that could overwrite the VideoReceiveStream render delay with a fixed value of 10ms on construction.  This wasn't a problem with the previous implementation (it would be now though) but seemed to me like the wrong place to be setting that value.

* Made the render delay value in VideoRenderFrames, const.

BUG=chromium:620232
TBR=mflodman

Review-Url: https://codereview.webrtc.org/2071473002
Cr-Commit-Position: refs/heads/master@{#13175}
2016-06-16 20:29:12 +00:00
17c3cddf9d Revert of Split IncomingVideoStream into two implementations, with smoothing and without. (patchset #23 id:430001 of https://codereview.webrtc.org/2035173002/ )
Reason for revert:
Reverting while we track down the issue on the Win10 bot.

Original issue's description:
> Split IncomingVideoStream into two implementations, with smoothing and without.
>
> This CL fixes an issue with the non-smoothing implementation where frames were delivered on the decoder thread.  No-smoothing is now done in a separate class that uses a TaskQueue.  The implementation may drop frames if the renderer doesn't keep up and it doesn't block the decoder thread.
>
> Further work done:
>
> * I added TODOs and documentation for VideoReceiveStream::OnFrame, where we today grab 5 locks.
>
> * I removed the Start/Stop methods from the IncomingVideoStream implementations.  Now, when an instance is created, it should be considered to be "running" and when it is deleted, it's "not running".  This saves on resources and also reduces the amount of locking required and I could remove one critical section altogether.
>
> * I changed the VideoStreamDecoder class to not depend on IncomingVideoStream but rather use the generic rtc::VideoSinkInterface<VideoFrame> interface.  This means that any implementation of that interface can be used and the decoder can be made to  just use the 'renderer' from the config.  Once we do that, we can decouple the IncomingVideoStream implementations from the decoder and VideoReceiveStream implementations and leave it up to the application for how to do smoothing.  The app can choose to use the Incoming* classes or roll its own (which may be preferable since applications often have their own scheduling mechanisms).
>
> * The non-smoothing IncomingVideoStream implementation currently allows only 1 outstanding pending frame.  If we exceed that, the current frame won't be delivered to the renderer and instead we deliver the next one (since when this happens, the renderer is falling behind).
>
> * The lifetime of the VideoStreamDecoder instance is now bound to Start/Stop in VideoReceiveStream and not all of the lifetime of VideoReceiveStream.
>
> * Fixed VideoStreamDecoder to unregister callbacks in the dtor that were registered in the ctor. (this was open to a use-after-free regression)
>
> * Delay and callback pointers are now passed via the ctors to the IncomingVideoStream classes.  The thread primitives in the IncomingVideoStream classes are also constructed/destructed at the same time as the owning object, which allowed me to remove one more lock.
>
> * Removed code in the VideoStreamDecoder that could overwrite the VideoReceiveStream render delay with a fixed value of 10ms on construction.  This wasn't a problem with the previous implementation (it would be now though) but seemed to me like the wrong place to be setting that value.
>
> * Made the render delay value in VideoRenderFrames, const.
>
> BUG=
>
> Committed: https://crrev.com/1c7eef652b0aa22d8ebb0bfe2b547094a794be22
> Cr-Commit-Position: refs/heads/master@{#13129}

TBR=mflodman@webrtc.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG=

Review-Url: https://codereview.webrtc.org/2061363002
Cr-Commit-Position: refs/heads/master@{#13146}
2016-06-14 23:04:48 +00:00
1c7eef652b Split IncomingVideoStream into two implementations, with smoothing and without.
This CL fixes an issue with the non-smoothing implementation where frames were delivered on the decoder thread.  No-smoothing is now done in a separate class that uses a TaskQueue.  The implementation may drop frames if the renderer doesn't keep up and it doesn't block the decoder thread.

Further work done:

* I added TODOs and documentation for VideoReceiveStream::OnFrame, where we today grab 5 locks.

* I removed the Start/Stop methods from the IncomingVideoStream implementations.  Now, when an instance is created, it should be considered to be "running" and when it is deleted, it's "not running".  This saves on resources and also reduces the amount of locking required and I could remove one critical section altogether.

* I changed the VideoStreamDecoder class to not depend on IncomingVideoStream but rather use the generic rtc::VideoSinkInterface<VideoFrame> interface.  This means that any implementation of that interface can be used and the decoder can be made to  just use the 'renderer' from the config.  Once we do that, we can decouple the IncomingVideoStream implementations from the decoder and VideoReceiveStream implementations and leave it up to the application for how to do smoothing.  The app can choose to use the Incoming* classes or roll its own (which may be preferable since applications often have their own scheduling mechanisms).

* The non-smoothing IncomingVideoStream implementation currently allows only 1 outstanding pending frame.  If we exceed that, the current frame won't be delivered to the renderer and instead we deliver the next one (since when this happens, the renderer is falling behind).

* The lifetime of the VideoStreamDecoder instance is now bound to Start/Stop in VideoReceiveStream and not all of the lifetime of VideoReceiveStream.

* Fixed VideoStreamDecoder to unregister callbacks in the dtor that were registered in the ctor. (this was open to a use-after-free regression)

* Delay and callback pointers are now passed via the ctors to the IncomingVideoStream classes.  The thread primitives in the IncomingVideoStream classes are also constructed/destructed at the same time as the owning object, which allowed me to remove one more lock.

* Removed code in the VideoStreamDecoder that could overwrite the VideoReceiveStream render delay with a fixed value of 10ms on construction.  This wasn't a problem with the previous implementation (it would be now though) but seemed to me like the wrong place to be setting that value.

* Made the render delay value in VideoRenderFrames, const.

BUG=

Review-Url: https://codereview.webrtc.org/2035173002
Cr-Commit-Position: refs/heads/master@{#13129}
2016-06-14 11:38:43 +00:00
718a763d59 Refactor scaling.
Introduce a new method I420Buffer::CropAndScale, and a static
convenience helper I420Buffer::CenterCropAndScale. Use them for almost
all scaling needs.

Delete the Scaler class and the cricket::VideoFrame::Stretch* methods.

BUG=webrtc:5682
R=pbos@webrtc.org, perkj@webrtc.org, stefan@webrtc.org

Review URL: https://codereview.webrtc.org/2020593002 .

Cr-Commit-Position: refs/heads/master@{#13110}
2016-06-13 11:06:14 +00:00
2a3892ac64 GN: Add common_audio_unittests and common_video_unittests
BUG=webrtc:5949
TESTED=Built and ran the tests on Mac.
NOTRY=True

Review-Url: https://codereview.webrtc.org/2040073002
Cr-Commit-Position: refs/heads/master@{#13061}
2016-06-08 08:28:15 +00:00
52033d6ea1 Add H264 bitstream rewriting to limit frame reordering marker in header
The VUI part an SPS may specify max_num_reorder_frames and
max_dec_frame_buffering. These may cause a decoder to buffer a number
of frame prior allowing decode, leading to delays, even if no frames
using such references (ie B-frames) are sent.

Because of this we update any SPS block emitted by the encoder.

Also, a bunch of refactoring of H264-related code to reduce code
duplication.

BUG=

Review-Url: https://codereview.webrtc.org/1979443004
Cr-Commit-Position: refs/heads/master@{#13010}
2016-06-02 09:43:38 +00:00
55a401e607 Move BitrateAdjuster into common_video
This CL moves BitrateAdjuster into common_video folder as it
was suggested on [0] such that it can be properly linked with
Chrome projects.

[0] https://codereview.chromium.org/1818903004/

BUG=500605

Review URL: https://codereview.webrtc.org/1914893005

Cr-Commit-Position: refs/heads/master@{#12515}
2016-04-26 19:55:10 +00:00
a96b60b3a6 Move frame_callback.h to common_video/include.
BUG=webrtc:4243
R=kjellander@webrtc.org, tommi@webrtc.org

Review URL: https://codereview.webrtc.org/1902543002

Cr-Commit-Position: refs/heads/master@{#12419}
2016-04-19 04:12:57 +00:00
7d06a8cfe4 Add CoreVideoFrameBuffer.
- Makes vt h264 decoder output CoreVideoFrameBuffer
- Makes iOS renderer convert frame buffer if it is not i420

BUG=

Review URL: https://codereview.webrtc.org/1853503003

Cr-Commit-Position: refs/heads/master@{#12224}
2016-04-04 21:10:47 +00:00
900f97534b H264: Improve FFmpeg decoder performance by using I420BufferPool.
Had to update I420BufferPool to allow zero-initializing buffers.

BUG=chromium:500605, chromium:468365, webrtc:5428

Review URL: https://codereview.webrtc.org/1645543003

Cr-Commit-Position: refs/heads/master@{#11505}
2016-02-05 16:08:39 +00:00
bab934bffe H.264 video codec support using OpenH264 (http://www.openh264.org/) for encoding and FFmpeg (https://www.ffmpeg.org/) for decoding.
It works on all platforms except Android and iOS (FFmpeg limitation).

Implemented behind compile time flags, off by default.
The plan is to have it enabled in Chrome (see bug), but not in Chromium/webrtc by default.

Flags to turn it on:
- rtc_use_h264 = true
- ffmpeg_branding = "Chrome" (or other brand that includes H.264 decoder)

Tests using H264:
- video_loopback --codec=H264
- screenshare_loopback --codec=H264
- video_engine_tests (EndToEndTest.SendsAndReceivesH264)

NOTRY=True
BUG=500605, 468365
BUG=https://bugs.chromium.org/p/webrtc/issues/detail?id=5424

Review URL: https://codereview.webrtc.org/1306813009

Cr-Commit-Position: refs/heads/master@{#11390}
2016-01-27 09:36:07 +00:00
6f8ce060a2 common_video: rename interface -> include
To avoid breaking downstream, the "interface" directories were copied
into a new "common_video/include" dir. The old headers got pragma
warnings added about deprecation (a very short deprecation since I plan
to remove them as soon downstream is updated).
The header guards are also identical to avoid mixing them up in the transition.

BUG=webrtc:5095
TESTED=Passing compile-trybots with --clobber flag:
git cl try --clobber --bot=win_compile_rel --bot=linux_compile_rel --bot=android_compile_rel --bot=mac_compile_rel --bot=ios_rel --bot=linux_gn_rel --bot=win_x64_gn_rel --bot=mac_x64_gn_rel --bot=android_gn_rel -m tryserver.webrtc

Review URL: https://codereview.webrtc.org/1418913006

Cr-Commit-Position: refs/heads/master@{#10659}
2015-11-16 21:52:31 +00:00
4765070b8d Rename I420VideoFrame to VideoFrame.
This is a mechanical change since it affects so many
files.
I420VideoFrame -> VideoFrame
and reformatted.

Rationale: in the next CL I420VideoFrame will
get an indication of Pixel Format (I420 for
starters) and of storage type: usually
UNOWNED, could be SHMEM, and in the near
future will be possibly TEXTURE. See
https://codereview.chromium.org/1154153003
for the change that happened in Cr.

BUG=4730, chromium:440843
R=jiayl@webrtc.org, niklas.enbom@webrtc.org, pthatcher@webrtc.org

Review URL: https://webrtc-codereview.appspot.com/52629004

Cr-Commit-Position: refs/heads/master@{#9339}
2015-05-30 00:21:56 +00:00
57e5fd2e60 PRESUBMIT: Improve PyLint check and add GN format check.
Add pylintrc file based on
https://code.google.com/p/chromium/codesearch#chromium/src/tools/perf/pylintrc
bit tightened up quite a bit (the one in depot_tools is far
more relaxed).

Remove a few excluded directories from pylint check and fixed/
suppressed all warnings generated.

Add GN format check + formatted all GN files using 'gn format'.
Cleanup redundant rules in tools/PRESUBMIT.py

TESTED=Ran 'git cl presubmit -vv', fixed the PyLint violations.
Ran it again with a modification in webrtc/build/webrtc.gni, formatted
all the GN files and ran it again.

R=henrika@webrtc.org, phoglund@webrtc.org

Review URL: https://webrtc-codereview.appspot.com/50069004

Cr-Commit-Position: refs/heads/master@{#9274}
2015-05-25 10:55:50 +00:00
9a63866272 Move IncomingVideoFrames to common_video/.
Permits using IncomingVideoFrame in VideoReceiveStream without depending
on VideoRender.

BUG=4588
R=mflodman@webrtc.org

Review URL: https://webrtc-codereview.appspot.com/49199004

Cr-Commit-Position: refs/heads/master@{#9184}
2015-05-13 11:28:04 +00:00
9bfe3daf73 Cleanup: Remove i420_video_frame.h header.
It is just a pass through to webrtc/video_frame.h. Updated the callers
to include webrtc/video_frame.h instead and removed i420_video_frame.h.

This should fix pbos' TODO in i420_video_frame.h.

Tested on Linux with the following command lines:

$ rm -rf out/
$ ./webrtc/build/gyp_webrtc
$ ninja -C out/Debug

BUG=None
TEST=see above
R=magjed@webrtc.org, pbos@webrtc.org, tommi@webrtc.org
TBR=tommi@webrtc.org

Review URL: https://webrtc-codereview.appspot.com/46819004

Patch from Thiago Farina <tfarina@chromium.org>.

Cr-Commit-Position: refs/heads/master@{#8973}
2015-04-10 10:52:15 +00:00
Per
9b3f56ea05 Reland "Remove usage of webrtc::NativeHandle since is just adds an extra level of indirection.""
This reverts commit e41d774c4d0a60066866fc2d0ae48dd0e839ff23.

Original code review: https://webrtc-codereview.appspot.com/43999004/
Reason for reland: There was nothing wrong with this cl as is, but it breaks chrome compatibility. We will now reland this and fix Chrome during roll.

Patset 1: Original cl.
Patchset 2: Removed more code that is no longer needed.

R=magjed@webrtc.org, pbos@webrtc.org
TBR=mflodman@webrtc.org

BUG=1128

Review URL: https://webrtc-codereview.appspot.com/45049004

Cr-Commit-Position: refs/heads/master@{#8956}
2015-04-09 11:44:19 +00:00
73d763e71f Add I420 buffer pool to avoid unnecessary allocations
Now when we don't use SwapFrame consistently anymore, we need to recycle allocations with a buffer pool instead. This CL adds a buffer pool class, and updates the vp8 decoder to use it. If this CL lands successfully I will update the other video producers as well.

BUG=1128
R=stefan@webrtc.org, tommi@webrtc.org

Review URL: https://webrtc-codereview.appspot.com/41189004

Cr-Commit-Position: refs/heads/master@{#8748}
git-svn-id: http://webrtc.googlecode.com/svn/trunk@8748 4adac7df-926f-26a2-2b94-8c16560cd09d
2015-03-17 11:41:15 +00:00
45cdcce5f5 Remove TextureVideoFrame
TextureVideoFrame is currently an empty shell that only provides a convenience constructor of I420VideoFrame with a texture buffer. This CL moves that constructor, and all unittests, of TextureVideoFrame into the base class. Then it's possible to completely remove TextureVideoFrame and all its files. Also, there is no point in having I420VideoFrame virtual anymore.

R=pbos@webrtc.org, perkj@webrtc.org, stefan@webrtc.org
TBR=mflodman

Review URL: https://webrtc-codereview.appspot.com/40229004

Cr-Commit-Position: refs/heads/master@{#8629}
git-svn-id: http://webrtc.googlecode.com/svn/trunk@8629 4adac7df-926f-26a2-2b94-8c16560cd09d
2015-03-06 10:41:47 +00:00
2386d6dd92 Revert 8599 "Revert 8580 "Unify underlying frame buffer in I420VideoFrame and...""
It's possible to build Chrome on Windows with this patch now.

BUG=1128

> This is unfortunately causing build problems in Chrome on Windows.

>> Unify underlying frame buffer in I420VideoFrame and WebRtcVideoFrame
>>
>> Currently, I420VideoFrame uses three webrtc::Plane to store pixel data, and WebRtcVideoFrame uses WebRtcVideoFrame::FrameBuffer/webrtc::VideoFrame. The two subclasses WebRtcTextureVideoFrame and TextureVideoFrame use a NativeHandle to store pixel data, and there is also a class WebRtcVideoRenderFrame that wraps an I420VideoFrame.
>>
>> This CL replaces these classes with a new interface VideoFrameBuffer that provides the common functionality. This makes it possible to remove deep frame copies between cricket::VideoFrame and I420VideoFrame.
>>
>> Some additional minor changes are:
>> * Disallow creation of 0x0 texture frames.
>> * Remove the half-implemented ref count functions in I420VideoFrame.
>> * Remove the Alias functionality in WebRtcVideoFrame
>>
>> The final goal is to eliminate all frame copies, but to limit the scope of this CL, some planned changes are postponed to follow-up CL:s (see planned changes in https://webrtc-codereview.appspot.com/38879004, or https://docs.google.com/document/d/1bxoJZNmlo-Z9GnQwIaWpEG6hDlL_W-bzka8Zb_K2NbA/preview). Specifically, this CL:
>> * Keeps empty subclasses WebRtcTextureVideoFrame and TextureVideoFrame, and just delegates the construction to the superclass.
>> * Keeps the deep copies from cricket::VideoFrame to I420VideoFrame.
>>
>> BUG=1128
>> R=mflodman@webrtc.org, pbos@webrtc.org, perkj@webrtc.org, tommi@webrtc.org
>>
>> Review URL: https://webrtc-codereview.appspot.com/42469004

R=pbos@webrtc.org
TBR=mflodman, pbos, perkj, tommi

Review URL: https://webrtc-codereview.appspot.com/45489004

Cr-Commit-Position: refs/heads/master@{#8616}
git-svn-id: http://webrtc.googlecode.com/svn/trunk@8616 4adac7df-926f-26a2-2b94-8c16560cd09d
2015-03-05 14:03:51 +00:00
1f94407319 Revert 8580 "Unify underlying frame buffer in I420VideoFrame and..."
This is unfortunately causing build problems in Chrome on Windows.

> Unify underlying frame buffer in I420VideoFrame and WebRtcVideoFrame
> 
> Currently, I420VideoFrame uses three webrtc::Plane to store pixel data, and WebRtcVideoFrame uses WebRtcVideoFrame::FrameBuffer/webrtc::VideoFrame. The two subclasses WebRtcTextureVideoFrame and TextureVideoFrame use a NativeHandle to store pixel data, and there is also a class WebRtcVideoRenderFrame that wraps an I420VideoFrame.
> 
> This CL replaces these classes with a new interface VideoFrameBuffer that provides the common functionality. This makes it possible to remove deep frame copies between cricket::VideoFrame and I420VideoFrame.
> 
> Some additional minor changes are:
> * Disallow creation of 0x0 texture frames.
> * Remove the half-implemented ref count functions in I420VideoFrame.
> * Remove the Alias functionality in WebRtcVideoFrame
> 
> The final goal is to eliminate all frame copies, but to limit the scope of this CL, some planned changes are postponed to follow-up CL:s (see planned changes in https://webrtc-codereview.appspot.com/38879004, or https://docs.google.com/document/d/1bxoJZNmlo-Z9GnQwIaWpEG6hDlL_W-bzka8Zb_K2NbA/preview). Specifically, this CL:
> * Keeps empty subclasses WebRtcTextureVideoFrame and TextureVideoFrame, and just delegates the construction to the superclass.
> * Keeps the deep copies from cricket::VideoFrame to I420VideoFrame.
> 
> BUG=1128
> R=mflodman@webrtc.org, pbos@webrtc.org, perkj@webrtc.org, tommi@webrtc.org
> 
> Review URL: https://webrtc-codereview.appspot.com/42469004

TBR=magjed@webrtc.org

Review URL: https://webrtc-codereview.appspot.com/42199005

Cr-Commit-Position: refs/heads/master@{#8599}
git-svn-id: http://webrtc.googlecode.com/svn/trunk@8599 4adac7df-926f-26a2-2b94-8c16560cd09d
2015-03-04 17:35:00 +00:00
c8895aa2f3 Unify underlying frame buffer in I420VideoFrame and WebRtcVideoFrame
Currently, I420VideoFrame uses three webrtc::Plane to store pixel data, and WebRtcVideoFrame uses WebRtcVideoFrame::FrameBuffer/webrtc::VideoFrame. The two subclasses WebRtcTextureVideoFrame and TextureVideoFrame use a NativeHandle to store pixel data, and there is also a class WebRtcVideoRenderFrame that wraps an I420VideoFrame.

This CL replaces these classes with a new interface VideoFrameBuffer that provides the common functionality. This makes it possible to remove deep frame copies between cricket::VideoFrame and I420VideoFrame.

Some additional minor changes are:
* Disallow creation of 0x0 texture frames.
* Remove the half-implemented ref count functions in I420VideoFrame.
* Remove the Alias functionality in WebRtcVideoFrame

The final goal is to eliminate all frame copies, but to limit the scope of this CL, some planned changes are postponed to follow-up CL:s (see planned changes in https://webrtc-codereview.appspot.com/38879004, or https://docs.google.com/document/d/1bxoJZNmlo-Z9GnQwIaWpEG6hDlL_W-bzka8Zb_K2NbA/preview). Specifically, this CL:
* Keeps empty subclasses WebRtcTextureVideoFrame and TextureVideoFrame, and just delegates the construction to the superclass.
* Keeps the deep copies from cricket::VideoFrame to I420VideoFrame.

BUG=1128
R=mflodman@webrtc.org, pbos@webrtc.org, perkj@webrtc.org, tommi@webrtc.org

Review URL: https://webrtc-codereview.appspot.com/42469004

Cr-Commit-Position: refs/heads/master@{#8580}
git-svn-id: http://webrtc.googlecode.com/svn/trunk@8580 4adac7df-926f-26a2-2b94-8c16560cd09d
2015-03-03 21:22:26 +00:00
ac2d27d9ae Fix style violations in common_types.h and config.h
Mostly, it's about moving constructors and descructors to the .cc
files, so that they won't be inlined everywhere.

The reason this CL is so big is that a lot of code was using
common_types.h without declaring a dependency on webrtc_common, which
broke the build once common_types.h started to depend on
common_types.cc.

BUG=163
R=kjellander@webrtc.org

Review URL: https://webrtc-codereview.appspot.com/26089004

Cr-Commit-Position: refs/heads/master@{#8516}
git-svn-id: http://webrtc.googlecode.com/svn/trunk@8516 4adac7df-926f-26a2-2b94-8c16560cd09d
2015-02-26 14:01:28 +00:00
ce22f13f0e GN: Changes for vp9, opus and direct trace
Corresponding GN changes for
https://webrtc-codereview.appspot.com/34099004/

BUG=4185
R=brettw@chromium.org

Review URL: https://webrtc-codereview.appspot.com/40669004

Cr-Commit-Position: refs/heads/master@{#8377}
git-svn-id: http://webrtc.googlecode.com/svn/trunk@8377 4adac7df-926f-26a2-2b94-8c16560cd09d
2015-02-16 12:47:45 +00:00
d7e34e1086 Make it easier to use external libyuv + cleanup GYP files.
It is now easier to use an external libyuv library.
Fix some GYP errors.
Remove the temporary webrtc_base target (depends on
https://codereview.chromium.org/865603002/ being landed
first).

BUG=4185
R=andresp@webrtc.org, andrew@webrtc.org, perkj@webrtc.org

Review URL: https://webrtc-codereview.appspot.com/39579004

git-svn-id: http://webrtc.googlecode.com/svn/trunk@8154 4adac7df-926f-26a2-2b94-8c16560cd09d
2015-01-26 19:17:26 +00:00
f21ea918ad GN: Add common configs to all targets.
This is needed to ensure we have the same build with GN
as with GYP, since GYP includes the common.gypi on a global level.
Several fixes has been needed in the past because some code have
been built without the right defines.

BUG=3441
R=brettw@chromium.org

Review URL: https://webrtc-codereview.appspot.com/28589004

git-svn-id: http://webrtc.googlecode.com/svn/trunk@7317 4adac7df-926f-26a2-2b94-8c16560cd09d
2014-09-28 17:37:22 +00:00
6d08ca6379 GN: Prefix WebRTC specific variables with "rtc_"
BUG=3441
TESTED=Trybots + Running GN in a Chromium checkout with
src/third_party/webrtc symlinked to the WebRTC checkout
with this CL applied, both with the default GN settings
and using: --args="os=\"android\" cpu_arch=\"arm\""

R=brettw@chromium.org

Review URL: https://webrtc-codereview.appspot.com/27379004

git-svn-id: http://webrtc.googlecode.com/svn/trunk@7095 4adac7df-926f-26a2-2b94-8c16560cd09d
2014-09-07 17:36:10 +00:00
42ee5b54b5 GN: Disable Chromium clang plugins for standalone build.
Now that WebRTC has rolled the chromium_revision past
http://crrev.com/284372 in r6784, clang has become the
default compiler. Since WebRTC standalone code doesn't
yet compile the Chromium Clang plugins enabled, this CL
disables them for the parts of the code that doesn't yet pass
compilation with them enabled.

The buildbots are using Goma which is not yet switched
over to Clang by default. That's why they're not red yet.

BUG=163
TEST=Passing compile locally on Linux using:
gn gen out/Debug --args="build_with_chromium=false is_debug=true" && ninja
-C out/Debug
gn gen out/Release --args="build_with_chromium=false is_debug=false" && ninja
-C out/Release
gn gen out/Default --args="build_with_chromium=false os=\"android\" cpu_arch=\"arm\" arm_version=7" && ninja -C out/Default

R=brettw@chromium.org

Review URL: https://webrtc-codereview.appspot.com/16279004

git-svn-id: http://webrtc.googlecode.com/svn/trunk@6966 4adac7df-926f-26a2-2b94-8c16560cd09d
2014-08-25 14:15:35 +00:00
11bea8977e GN: Implement BUILD.gn for common_video.
This adds copying of Chromium's third_party/BUILD.gn
to acommondate libyuv's BUILD.gn that imports the 'jpeg'
config from that file.

BUG=3441
TEST=trybots + local compile passing with:
gn gen out/Default --args="build_with_chromium=false" && ninja -C out/Default
gn gen out/Default --args="build_with_chromium=false build_libyuv=false" && ninja -C out/Default

R=niklas.enbom@webrtc.org, pbos@webrtc.org

Review URL: https://webrtc-codereview.appspot.com/14839004

git-svn-id: http://webrtc.googlecode.com/svn/trunk@6595 4adac7df-926f-26a2-2b94-8c16560cd09d
2014-07-03 17:04:12 +00:00