Commit Graph

46 Commits

Author SHA1 Message Date
4e417b242a Reland of Switch to use SequencedTaskChecker instead of ThreadChecker where needed.
(patchset #1 id:1 of https://codereview.webrtc.org/2149553002/ )"
This reverts commit efd902cb1d9bbd81247a3e168f2080beae761d78.

Originally reviewed in https://codereview.webrtc.org/2149553002

The uptream problem should be fixed by https://codereview.webrtc.org/2145393003/

BUG=webrtc:5687
TBR=tommi@webrtc.org

Review-Url: https://codereview.webrtc.org/2152013002
Cr-Commit-Position: refs/heads/master@{#13483}
2016-07-15 06:36:00 +00:00
efd902cb1d Revert of Switch to use SequencedTaskChecker instead of ThreadChecker where needed. (patchset #1 id:1 of https://codereview.webrtc.org/2149553002/ )
Reason for revert:
Breaks Chrome FYI

https://build.chromium.org/p/chromium.webrtc.fyi/builders/Linux%20Builder/builds/8187/steps/compile/logs/stdio

Original issue's description:
> Switch to use SequencedTaskChecker instead of ThreadChecker where needed.
> This cl is in preparation for https://codereview.webrtc.org/2060403002/ Add task queue to Call.
> In the coming cl the video_sender, and i420_buffer_pool will be used on a task queue and therefore SequencedTaskChecker is needed instead of a ThreadChecker.
>
> BUG=webrtc:5687
>
> Committed: https://crrev.com/ec7cef854d85be2d9fd5b1a4e2c59a67feb662eb
> Cr-Commit-Position: refs/heads/master@{#13474}

TBR=tommi@webrtc.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG=webrtc:5687

Review-Url: https://codereview.webrtc.org/2150843003
Cr-Commit-Position: refs/heads/master@{#13475}
2016-07-14 09:33:25 +00:00
ec7cef854d Switch to use SequencedTaskChecker instead of ThreadChecker where needed.
This cl is in preparation for https://codereview.webrtc.org/2060403002/ Add task queue to Call.
In the coming cl the video_sender, and i420_buffer_pool will be used on a task queue and therefore SequencedTaskChecker is needed instead of a ThreadChecker.

BUG=webrtc:5687

Review-Url: https://codereview.webrtc.org/2149553002
Cr-Commit-Position: refs/heads/master@{#13474}
2016-07-14 08:32:09 +00:00
02bafc6379 Add a race-checking mechanism.
Permits CHECKing/DCHECKing that methods are being accessed in a
thread-safe manner, even if they are not used by one single thread
(thread pools such as VideoToolbox OK).

BUG=
R=danilchap@webrtc.org, tommi@webrtc.org

Review URL: https://codereview.webrtc.org/2097403002 .

Cr-Commit-Position: refs/heads/master@{#13358}
2016-07-01 10:45:29 +00:00
7893e9a267 Remove a thread checker for a decoder thread from IncomingVideoStream.
The specific decoder thread may vary when using VideoToolbox.

BUG=webrtc:6051
TBR=pbos@webrtc.org

Review URL: https://codereview.webrtc.org/2100083002 .

Cr-Commit-Position: refs/heads/master@{#13292}
2016-06-27 07:54:58 +00:00
1e6bbe4538 Delete deprecated VideoFrameBuffer methods.
(Reland of part of https://codereview.webrtc.org/2065733003/).

BUG=webrtc:5682

Review-Url: https://codereview.webrtc.org/2088753002
Cr-Commit-Position: refs/heads/master@{#13235}
2016-06-21 10:59:32 +00:00
2e82f3821f Reland of Split IncomingVideoStream into two implementations, with smoothing and without. (patchset #1 id:1 of https://codereview.webrtc.org/2084873002/ )
Reason for revert:
Reverting the revert.  This change is not related to the failure on the Windows FYI bots.  The cause of the failure has been reverted in Chromium:
https://codereview.chromium.org/2081653004/

Original issue's description:
> Revert of Split IncomingVideoStream into two implementations, with smoothing and without. (patchset #5 id:340001 of https://codereview.webrtc.org/2078873002/ )
>
> Reason for revert:
> Breaks chromium.webrtc.fyi
>
> https://uberchromegw.corp.google.com/i/chromium.webrtc.fyi/builders/Win7%20Tester/builds/4719
> https://uberchromegw.corp.google.com/i/chromium.webrtc.fyi/builders/Win10%20Tester/builds/3120
>
> Original issue's description:
> > Reland of IncomingVideoStream refactoring.
> > This reland does not contain the non-smoothing part of the original implementation.  Instead, when smoothing is turned off, frame callbacks run on the decoder thread, as they did before.  This code path is used in Chrome.  As far as Chrome goes, the difference now is that there won't be an instance of IncomingVideoStream in between the decoder and the callback (i.e. fewer locks).  Other than that, no change for Chrome.
> >
> > Original issue's description (with non-smoothing references removed):
> >
> > Split IncomingVideoStream into two implementations, with smoothing and without.
> >
> > * Added TODOs and documentation for VideoReceiveStream::OnFrame, where we today grab 6 locks.
> >
> > * Removed the Start/Stop methods from the IncomingVideoStream implementations.  Now, when an instance is created, it should be considered to be "running" and when it is deleted, it's "not running".  This saves on resources and also reduces the amount of locking required and I could remove one critical section altogether.
> >
> > * Changed the VideoStreamDecoder class to not depend on IncomingVideoStream but rather use the generic rtc::VideoSinkInterface<VideoFrame> interface.  This means that any implementation of that interface can be used and the decoder can be made to  just use the 'renderer' from the config.  Once we do that, we can decouple the IncomingVideoStream implementations from the decoder and VideoReceiveStream implementations and leave it up to the application for how to do smoothing.  The app can choose to use the Incoming* classes or roll its own (which may be preferable since applications often have their own scheduling mechanisms).
> >
> > * The lifetime of the VideoStreamDecoder instance is now bound to Start/Stop in VideoReceiveStream and not all of the lifetime of VideoReceiveStream.
> >
> > * Fixed VideoStreamDecoder to unregister callbacks in the dtor that were registered in the ctor. (this was open to a use-after-free regression)
> >
> > * Delay and callback pointers are now passed via the ctors to the IncomingVideoStream classes.  The thread primitives in the IncomingVideoStream classes are also constructed/destructed at the same time as the owning object, which allowed me to remove one more lock.
> >
> > * Removed code in the VideoStreamDecoder that could overwrite the VideoReceiveStream render delay with a fixed value of 10ms on construction.  This wasn't a problem with the previous implementation (it would be now though) but seemed to me like the wrong place to be setting that value.
> >
> > * Made the render delay value in VideoRenderFrames, const.
> >
> > BUG=chromium:620232
> > R=mflodman@webrtc.org, nisse@webrtc.org
> >
> > Committed: https://crrev.com/884c336c345d988974c2a69cea402b0fb8b07a63
> > Cr-Commit-Position: refs/heads/master@{#13219}
>
> TBR=nisse@webrtc.org,philipel@webrtc.org,mflodman@webrtc.org,tommi@webrtc.org
> # Skipping CQ checks because original CL landed less than 1 days ago.
> NOPRESUBMIT=true
> NOTREECHECKS=true
> NOTRY=true
> BUG=chromium:620232
>
> Committed: https://crrev.com/a536bfe70de38fe877245317a7f0b00bcf69cbd0
> Cr-Commit-Position: refs/heads/master@{#13229}

TBR=nisse@webrtc.org,philipel@webrtc.org,mflodman@webrtc.org,sakal@webrtc.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG=chromium:620232

Review-Url: https://codereview.webrtc.org/2089613002
Cr-Commit-Position: refs/heads/master@{#13230}
2016-06-21 07:26:48 +00:00
a536bfe70d Revert of Split IncomingVideoStream into two implementations, with smoothing and without. (patchset #5 id:340001 of https://codereview.webrtc.org/2078873002/ )
Reason for revert:
Breaks chromium.webrtc.fyi

https://uberchromegw.corp.google.com/i/chromium.webrtc.fyi/builders/Win7%20Tester/builds/4719
https://uberchromegw.corp.google.com/i/chromium.webrtc.fyi/builders/Win10%20Tester/builds/3120

Original issue's description:
> Reland of IncomingVideoStream refactoring.
> This reland does not contain the non-smoothing part of the original implementation.  Instead, when smoothing is turned off, frame callbacks run on the decoder thread, as they did before.  This code path is used in Chrome.  As far as Chrome goes, the difference now is that there won't be an instance of IncomingVideoStream in between the decoder and the callback (i.e. fewer locks).  Other than that, no change for Chrome.
>
> Original issue's description (with non-smoothing references removed):
>
> Split IncomingVideoStream into two implementations, with smoothing and without.
>
> * Added TODOs and documentation for VideoReceiveStream::OnFrame, where we today grab 6 locks.
>
> * Removed the Start/Stop methods from the IncomingVideoStream implementations.  Now, when an instance is created, it should be considered to be "running" and when it is deleted, it's "not running".  This saves on resources and also reduces the amount of locking required and I could remove one critical section altogether.
>
> * Changed the VideoStreamDecoder class to not depend on IncomingVideoStream but rather use the generic rtc::VideoSinkInterface<VideoFrame> interface.  This means that any implementation of that interface can be used and the decoder can be made to  just use the 'renderer' from the config.  Once we do that, we can decouple the IncomingVideoStream implementations from the decoder and VideoReceiveStream implementations and leave it up to the application for how to do smoothing.  The app can choose to use the Incoming* classes or roll its own (which may be preferable since applications often have their own scheduling mechanisms).
>
> * The lifetime of the VideoStreamDecoder instance is now bound to Start/Stop in VideoReceiveStream and not all of the lifetime of VideoReceiveStream.
>
> * Fixed VideoStreamDecoder to unregister callbacks in the dtor that were registered in the ctor. (this was open to a use-after-free regression)
>
> * Delay and callback pointers are now passed via the ctors to the IncomingVideoStream classes.  The thread primitives in the IncomingVideoStream classes are also constructed/destructed at the same time as the owning object, which allowed me to remove one more lock.
>
> * Removed code in the VideoStreamDecoder that could overwrite the VideoReceiveStream render delay with a fixed value of 10ms on construction.  This wasn't a problem with the previous implementation (it would be now though) but seemed to me like the wrong place to be setting that value.
>
> * Made the render delay value in VideoRenderFrames, const.
>
> BUG=chromium:620232
> R=mflodman@webrtc.org, nisse@webrtc.org
>
> Committed: https://crrev.com/884c336c345d988974c2a69cea402b0fb8b07a63
> Cr-Commit-Position: refs/heads/master@{#13219}

TBR=nisse@webrtc.org,philipel@webrtc.org,mflodman@webrtc.org,tommi@webrtc.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG=chromium:620232

Review-Url: https://codereview.webrtc.org/2084873002
Cr-Commit-Position: refs/heads/master@{#13229}
2016-06-21 07:08:58 +00:00
884c336c34 Reland of IncomingVideoStream refactoring.
This reland does not contain the non-smoothing part of the original implementation.  Instead, when smoothing is turned off, frame callbacks run on the decoder thread, as they did before.  This code path is used in Chrome.  As far as Chrome goes, the difference now is that there won't be an instance of IncomingVideoStream in between the decoder and the callback (i.e. fewer locks).  Other than that, no change for Chrome.

Original issue's description (with non-smoothing references removed):

Split IncomingVideoStream into two implementations, with smoothing and without.

* Added TODOs and documentation for VideoReceiveStream::OnFrame, where we today grab 6 locks.

* Removed the Start/Stop methods from the IncomingVideoStream implementations.  Now, when an instance is created, it should be considered to be "running" and when it is deleted, it's "not running".  This saves on resources and also reduces the amount of locking required and I could remove one critical section altogether.

* Changed the VideoStreamDecoder class to not depend on IncomingVideoStream but rather use the generic rtc::VideoSinkInterface<VideoFrame> interface.  This means that any implementation of that interface can be used and the decoder can be made to  just use the 'renderer' from the config.  Once we do that, we can decouple the IncomingVideoStream implementations from the decoder and VideoReceiveStream implementations and leave it up to the application for how to do smoothing.  The app can choose to use the Incoming* classes or roll its own (which may be preferable since applications often have their own scheduling mechanisms).

* The lifetime of the VideoStreamDecoder instance is now bound to Start/Stop in VideoReceiveStream and not all of the lifetime of VideoReceiveStream.

* Fixed VideoStreamDecoder to unregister callbacks in the dtor that were registered in the ctor. (this was open to a use-after-free regression)

* Delay and callback pointers are now passed via the ctors to the IncomingVideoStream classes.  The thread primitives in the IncomingVideoStream classes are also constructed/destructed at the same time as the owning object, which allowed me to remove one more lock.

* Removed code in the VideoStreamDecoder that could overwrite the VideoReceiveStream render delay with a fixed value of 10ms on construction.  This wasn't a problem with the previous implementation (it would be now though) but seemed to me like the wrong place to be setting that value.

* Made the render delay value in VideoRenderFrames, const.

BUG=chromium:620232
R=mflodman@webrtc.org, nisse@webrtc.org

Review URL: https://codereview.webrtc.org/2078873002 .

Cr-Commit-Position: refs/heads/master@{#13219}
2016-06-20 17:43:10 +00:00
ac62bd4a3b Rewrite CreateBlackFrame in webrtcvideoengine.
Don't use VideoFrameBuffer::MutableDataY and friends, instead, use
I420Buffer::SetToBlack.

Also introduce static method I420Buffer::Create, to create an object and
return a scoped_refptr.

TBR=marpan@webrtc.org # Trivial change to video_denoiser.cc
BUG=webrtc:5921

Review-Url: https://codereview.webrtc.org/2078943002
Cr-Commit-Position: refs/heads/master@{#13212}
2016-06-20 10:39:00 +00:00
72e735d386 Revert of Delete unused and almost unused frame-related methods. (patchset #12 id:220001 of https://codereview.webrtc.org/2065733003/ )
Reason for revert:
Breaks downstream applications which inherits webrtc::VideoFrameBuffer and tries to override deleted methods data(), stride() and MutableData().

Original issue's description:
> Delete unused and almost unused frame-related methods.
>
> webrtc::VideoFrame::set_video_frame_buffer
> webrtc::VideoFrame::ConvertNativeToI420Frame
>
> cricket::WebRtcVideoFrame::InitToBlack
>
> VideoFrameBuffer::data
> VideoFrameBuffer::stride
> VideoFrameBuffer::MutableData
>
> TBR=tkchin@webrtc.org # Refactoring affecting RTCVideoFrame
> BUG=webrtc:5682
>
> Committed: https://crrev.com/76270de4bc2dac188f10f805e6e2fb86693ef864
> Cr-Commit-Position: refs/heads/master@{#13183}

TBR=perkj@webrtc.org,pbos@webrtc.org,marpan@webrtc.org,tkchin@webrtc.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG=webrtc:5682

Review-Url: https://codereview.webrtc.org/2076113002
Cr-Commit-Position: refs/heads/master@{#13184}
2016-06-17 09:55:23 +00:00
76270de4bc Delete unused and almost unused frame-related methods.
webrtc::VideoFrame::set_video_frame_buffer
webrtc::VideoFrame::ConvertNativeToI420Frame

cricket::WebRtcVideoFrame::InitToBlack

VideoFrameBuffer::data
VideoFrameBuffer::stride
VideoFrameBuffer::MutableData

TBR=tkchin@webrtc.org # Refactoring affecting RTCVideoFrame
BUG=webrtc:5682

Review-Url: https://codereview.webrtc.org/2065733003
Cr-Commit-Position: refs/heads/master@{#13183}
2016-06-17 09:00:19 +00:00
6af2e86b46 Refactor VideoDenoiser to work with I420Buffer, not VideoFrame.
BUG=webrtc:5921
R=jackychen@webrtc.org, marpan@webrtc.org

Review URL: https://codereview.webrtc.org/2005733003 .

Cr-Commit-Position: refs/heads/master@{#13179}
2016-06-17 07:12:55 +00:00
8e8222d0d2 Revert of Split IncomingVideoStream into two implementations, with smoothing and without. (patchset #4 id:290001 of https://codereview.webrtc.org/2071473002/ )
Reason for revert:
Reverting again.  The perf regression does not seem to be related to dropping frames.

Original issue's description:
> Reland of Split IncomingVideoStream into two implementations, with smoothing and without.
>
> Original issue's description:
>
> Split IncomingVideoStream into two implementations, with smoothing and without.
>
> This CL fixes an issue with the non-smoothing implementation where frames were delivered on the decoder thread.  No-smoothing is now done in a separate class that uses a TaskQueue.  The implementation may drop frames if the renderer doesn't keep up and it doesn't block the decoder thread.
>
> Further work done:
>
> * I added TODOs and documentation for VideoReceiveStream::OnFrame, where we today grab 5 locks.
>
> * I removed the Start/Stop methods from the IncomingVideoStream implementations.  Now, when an instance is created, it should be considered to be "running" and when it is deleted, it's "not running".  This saves on resources and also reduces the amount of locking required and I could remove one critical section altogether.
>
> * I changed the VideoStreamDecoder class to not depend on IncomingVideoStream but rather use the generic rtc::VideoSinkInterface<VideoFrame> interface.  This means that any implementation of that interface can be used and the decoder can be made to  just use the 'renderer' from the config.  Once we do that, we can decouple the IncomingVideoStream implementations from the decoder and VideoReceiveStream implementations and leave it up to the application for how to do smoothing.  The app can choose to use the Incoming* classes or roll its own (which may be preferable since applications often have their own scheduling mechanisms).
>
> * The non-smoothing IncomingVideoStream implementation currently allows only 1 outstanding pending frame.  If we exceed that, the current frame won't be delivered to the renderer and instead we deliver the next one (since when this happens, the renderer is falling behind).
>
> * The lifetime of the VideoStreamDecoder instance is now bound to Start/Stop in VideoReceiveStream and not all of the lifetime of VideoReceiveStream.
>
> * Fixed VideoStreamDecoder to unregister callbacks in the dtor that were registered in the ctor. (this was open to a use-after-free regression)
>
> * Delay and callback pointers are now passed via the ctors to the IncomingVideoStream classes.  The thread primitives in the IncomingVideoStream classes are also constructed/destructed at the same time as the owning object, which allowed me to remove one more lock.
>
> * Removed code in the VideoStreamDecoder that could overwrite the VideoReceiveStream render delay with a fixed value of 10ms on construction.  This wasn't a problem with the previous implementation (it would be now though) but seemed to me like the wrong place to be setting that value.
>
> * Made the render delay value in VideoRenderFrames, const.
>
> BUG=chromium:620232
> TBR=mflodman
>
> Committed: https://crrev.com/e03f8787377bbc03a4e00184bb14b7561b108cbb
> Cr-Commit-Position: refs/heads/master@{#13175}

TBR=mflodman@webrtc.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG=chromium:620232

Review-Url: https://codereview.webrtc.org/2071093002
Cr-Commit-Position: refs/heads/master@{#13176}
2016-06-16 22:44:11 +00:00
e03f878737 Reland of Split IncomingVideoStream into two implementations, with smoothing and without.
Original issue's description:

Split IncomingVideoStream into two implementations, with smoothing and without.

This CL fixes an issue with the non-smoothing implementation where frames were delivered on the decoder thread.  No-smoothing is now done in a separate class that uses a TaskQueue.  The implementation may drop frames if the renderer doesn't keep up and it doesn't block the decoder thread.

Further work done:

* I added TODOs and documentation for VideoReceiveStream::OnFrame, where we today grab 5 locks.

* I removed the Start/Stop methods from the IncomingVideoStream implementations.  Now, when an instance is created, it should be considered to be "running" and when it is deleted, it's "not running".  This saves on resources and also reduces the amount of locking required and I could remove one critical section altogether.

* I changed the VideoStreamDecoder class to not depend on IncomingVideoStream but rather use the generic rtc::VideoSinkInterface<VideoFrame> interface.  This means that any implementation of that interface can be used and the decoder can be made to  just use the 'renderer' from the config.  Once we do that, we can decouple the IncomingVideoStream implementations from the decoder and VideoReceiveStream implementations and leave it up to the application for how to do smoothing.  The app can choose to use the Incoming* classes or roll its own (which may be preferable since applications often have their own scheduling mechanisms).

* The non-smoothing IncomingVideoStream implementation currently allows only 1 outstanding pending frame.  If we exceed that, the current frame won't be delivered to the renderer and instead we deliver the next one (since when this happens, the renderer is falling behind).

* The lifetime of the VideoStreamDecoder instance is now bound to Start/Stop in VideoReceiveStream and not all of the lifetime of VideoReceiveStream.

* Fixed VideoStreamDecoder to unregister callbacks in the dtor that were registered in the ctor. (this was open to a use-after-free regression)

* Delay and callback pointers are now passed via the ctors to the IncomingVideoStream classes.  The thread primitives in the IncomingVideoStream classes are also constructed/destructed at the same time as the owning object, which allowed me to remove one more lock.

* Removed code in the VideoStreamDecoder that could overwrite the VideoReceiveStream render delay with a fixed value of 10ms on construction.  This wasn't a problem with the previous implementation (it would be now though) but seemed to me like the wrong place to be setting that value.

* Made the render delay value in VideoRenderFrames, const.

BUG=chromium:620232
TBR=mflodman

Review-Url: https://codereview.webrtc.org/2071473002
Cr-Commit-Position: refs/heads/master@{#13175}
2016-06-16 20:29:12 +00:00
17c3cddf9d Revert of Split IncomingVideoStream into two implementations, with smoothing and without. (patchset #23 id:430001 of https://codereview.webrtc.org/2035173002/ )
Reason for revert:
Reverting while we track down the issue on the Win10 bot.

Original issue's description:
> Split IncomingVideoStream into two implementations, with smoothing and without.
>
> This CL fixes an issue with the non-smoothing implementation where frames were delivered on the decoder thread.  No-smoothing is now done in a separate class that uses a TaskQueue.  The implementation may drop frames if the renderer doesn't keep up and it doesn't block the decoder thread.
>
> Further work done:
>
> * I added TODOs and documentation for VideoReceiveStream::OnFrame, where we today grab 5 locks.
>
> * I removed the Start/Stop methods from the IncomingVideoStream implementations.  Now, when an instance is created, it should be considered to be "running" and when it is deleted, it's "not running".  This saves on resources and also reduces the amount of locking required and I could remove one critical section altogether.
>
> * I changed the VideoStreamDecoder class to not depend on IncomingVideoStream but rather use the generic rtc::VideoSinkInterface<VideoFrame> interface.  This means that any implementation of that interface can be used and the decoder can be made to  just use the 'renderer' from the config.  Once we do that, we can decouple the IncomingVideoStream implementations from the decoder and VideoReceiveStream implementations and leave it up to the application for how to do smoothing.  The app can choose to use the Incoming* classes or roll its own (which may be preferable since applications often have their own scheduling mechanisms).
>
> * The non-smoothing IncomingVideoStream implementation currently allows only 1 outstanding pending frame.  If we exceed that, the current frame won't be delivered to the renderer and instead we deliver the next one (since when this happens, the renderer is falling behind).
>
> * The lifetime of the VideoStreamDecoder instance is now bound to Start/Stop in VideoReceiveStream and not all of the lifetime of VideoReceiveStream.
>
> * Fixed VideoStreamDecoder to unregister callbacks in the dtor that were registered in the ctor. (this was open to a use-after-free regression)
>
> * Delay and callback pointers are now passed via the ctors to the IncomingVideoStream classes.  The thread primitives in the IncomingVideoStream classes are also constructed/destructed at the same time as the owning object, which allowed me to remove one more lock.
>
> * Removed code in the VideoStreamDecoder that could overwrite the VideoReceiveStream render delay with a fixed value of 10ms on construction.  This wasn't a problem with the previous implementation (it would be now though) but seemed to me like the wrong place to be setting that value.
>
> * Made the render delay value in VideoRenderFrames, const.
>
> BUG=
>
> Committed: https://crrev.com/1c7eef652b0aa22d8ebb0bfe2b547094a794be22
> Cr-Commit-Position: refs/heads/master@{#13129}

TBR=mflodman@webrtc.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG=

Review-Url: https://codereview.webrtc.org/2061363002
Cr-Commit-Position: refs/heads/master@{#13146}
2016-06-14 23:04:48 +00:00
1c7eef652b Split IncomingVideoStream into two implementations, with smoothing and without.
This CL fixes an issue with the non-smoothing implementation where frames were delivered on the decoder thread.  No-smoothing is now done in a separate class that uses a TaskQueue.  The implementation may drop frames if the renderer doesn't keep up and it doesn't block the decoder thread.

Further work done:

* I added TODOs and documentation for VideoReceiveStream::OnFrame, where we today grab 5 locks.

* I removed the Start/Stop methods from the IncomingVideoStream implementations.  Now, when an instance is created, it should be considered to be "running" and when it is deleted, it's "not running".  This saves on resources and also reduces the amount of locking required and I could remove one critical section altogether.

* I changed the VideoStreamDecoder class to not depend on IncomingVideoStream but rather use the generic rtc::VideoSinkInterface<VideoFrame> interface.  This means that any implementation of that interface can be used and the decoder can be made to  just use the 'renderer' from the config.  Once we do that, we can decouple the IncomingVideoStream implementations from the decoder and VideoReceiveStream implementations and leave it up to the application for how to do smoothing.  The app can choose to use the Incoming* classes or roll its own (which may be preferable since applications often have their own scheduling mechanisms).

* The non-smoothing IncomingVideoStream implementation currently allows only 1 outstanding pending frame.  If we exceed that, the current frame won't be delivered to the renderer and instead we deliver the next one (since when this happens, the renderer is falling behind).

* The lifetime of the VideoStreamDecoder instance is now bound to Start/Stop in VideoReceiveStream and not all of the lifetime of VideoReceiveStream.

* Fixed VideoStreamDecoder to unregister callbacks in the dtor that were registered in the ctor. (this was open to a use-after-free regression)

* Delay and callback pointers are now passed via the ctors to the IncomingVideoStream classes.  The thread primitives in the IncomingVideoStream classes are also constructed/destructed at the same time as the owning object, which allowed me to remove one more lock.

* Removed code in the VideoStreamDecoder that could overwrite the VideoReceiveStream render delay with a fixed value of 10ms on construction.  This wasn't a problem with the previous implementation (it would be now though) but seemed to me like the wrong place to be setting that value.

* Made the render delay value in VideoRenderFrames, const.

BUG=

Review-Url: https://codereview.webrtc.org/2035173002
Cr-Commit-Position: refs/heads/master@{#13129}
2016-06-14 11:38:43 +00:00
718a763d59 Refactor scaling.
Introduce a new method I420Buffer::CropAndScale, and a static
convenience helper I420Buffer::CenterCropAndScale. Use them for almost
all scaling needs.

Delete the Scaler class and the cricket::VideoFrame::Stretch* methods.

BUG=webrtc:5682
R=pbos@webrtc.org, perkj@webrtc.org, stefan@webrtc.org

Review URL: https://codereview.webrtc.org/2020593002 .

Cr-Commit-Position: refs/heads/master@{#13110}
2016-06-13 11:06:14 +00:00
51e60305e6 Update RateStatistics to handle too-little-data case.
To avoid the case where a single data point or too short window is used,
causing bad behavior due to bad stats, update RateStatistics to return
an Optional rather than a plain rate.

There was also a strange off by one bug where the rate was slightly
overestimated (N + 1 buckets, N ms time window).

These changes requires updates to a number of places, and may very well
cause seeming perf regressions (but the stats were probablty more wrong
previously).

BUG=
R=mflodman@webrtc.org, stefan@webrtc.org

Review URL: https://codereview.webrtc.org/2029593002 .

Cr-Commit-Position: refs/heads/master@{#13103}
2016-06-10 20:13:33 +00:00
efec5902a5 Reland of New method I420Buffer::SetToBlack. (patchset #1 id:1 of https://codereview.webrtc.org/2049023002/ )
Reason for revert:
Plan to reland with InitToBlack kept, to be able to update Chrome to use the new I420Buffer::SetToBlack method.

Original issue's description:
> Revert of New static method I420Buffer::SetToBlack. (patchset #4 id:60001 of https://codereview.webrtc.org/2029273004/ )
>
> Reason for revert:
> Breaks chrome, in particular, the tests in
>
> media_stream_remote_video_source_unittest.cc
>
> use the InitToBlack method which is being deleted.
>
> Original issue's description:
> > New static method I420Buffer::SetToBlack.
> >
> > Replaces cricket::VideoFrame::SetToBlack and
> > cricket::WebRtcVideoFrame::InitToBlack, which are deleted.
> >
> > Refactors the black frame logic in VideoBroadcaster, and a few of the
> > tests.
> >
> > BUG=webrtc:5682
> >
> > Committed: https://crrev.com/663f9e2ddc86e813f6db04ba2cf5ac1ed9e7ef67
> > Cr-Commit-Position: refs/heads/master@{#13063}
>
> TBR=perkj@webrtc.org,pbos@webrtc.org
> # Skipping CQ checks because original CL landed less than 1 days ago.
> NOPRESUBMIT=true
> NOTREECHECKS=true
> NOTRY=true
> BUG=webrtc:5682
>
> Committed: https://crrev.com/271d74078894bb24f454eb31b77e4ce38097a2fa
> Cr-Commit-Position: refs/heads/master@{#13065}

TBR=perkj@webrtc.org,pbos@webrtc.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG=webrtc:5682

Review-Url: https://codereview.webrtc.org/2049513005
Cr-Commit-Position: refs/heads/master@{#13083}
2016-06-09 07:31:46 +00:00
271d740788 Revert of New static method I420Buffer::SetToBlack. (patchset #4 id:60001 of https://codereview.webrtc.org/2029273004/ )
Reason for revert:
Breaks chrome, in particular, the tests in

media_stream_remote_video_source_unittest.cc

use the InitToBlack method which is being deleted.

Original issue's description:
> New static method I420Buffer::SetToBlack.
>
> Replaces cricket::VideoFrame::SetToBlack and
> cricket::WebRtcVideoFrame::InitToBlack, which are deleted.
>
> Refactors the black frame logic in VideoBroadcaster, and a few of the
> tests.
>
> BUG=webrtc:5682
>
> Committed: https://crrev.com/663f9e2ddc86e813f6db04ba2cf5ac1ed9e7ef67
> Cr-Commit-Position: refs/heads/master@{#13063}

TBR=perkj@webrtc.org,pbos@webrtc.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG=webrtc:5682

Review-Url: https://codereview.webrtc.org/2049023002
Cr-Commit-Position: refs/heads/master@{#13065}
2016-06-08 12:21:02 +00:00
663f9e2ddc New static method I420Buffer::SetToBlack.
Replaces cricket::VideoFrame::SetToBlack and
cricket::WebRtcVideoFrame::InitToBlack, which are deleted.

Refactors the black frame logic in VideoBroadcaster, and a few of the
tests.

BUG=webrtc:5682

Review-Url: https://codereview.webrtc.org/2029273004
Cr-Commit-Position: refs/heads/master@{#13063}
2016-06-08 11:26:27 +00:00
ba986bfbaf Remove dead code in IncomingVideoStream that's just causing contention.
BUG=

Review-Url: https://codereview.webrtc.org/2040763002
Cr-Commit-Position: refs/heads/master@{#13053}
2016-06-06 19:21:21 +00:00
5724b7317e Remove unused render callback method+pointer in IncomingVideoStream.
Review-Url: https://codereview.webrtc.org/2032003002
Cr-Commit-Position: refs/heads/master@{#13009}
2016-06-02 09:39:12 +00:00
d591e3fcf3 Delete IsMutable and IsExclusive methods.
This affects the webrtc::VideoFrameBuffer and cricket::VideoFrame
classes.

To make this work, VideoFrameFactory is changed to use an
I420BufferPool rather than a plain VideoFrame to cache allocated
frames.

The I420BufferPool is reorganized to return an I420Buffer,
rather than a proxy object.

BUG=webrtc:5921, webrtc:5682

Review-Url: https://codereview.webrtc.org/2009193002
Cr-Commit-Position: refs/heads/master@{#12919}
2016-05-26 13:50:00 +00:00
30f118effd This cl deletes the class webrtc::VideoRendererCallback.
Replaced by VideoSinkInterface instead.

Also delete stream_id property of IncomingVideoStream.

BUG=webrtc:5426

Review-Url: https://codereview.webrtc.org/1813173002
Cr-Commit-Position: refs/heads/master@{#12602}
2016-05-03 08:09:17 +00:00
55a401e607 Move BitrateAdjuster into common_video
This CL moves BitrateAdjuster into common_video folder as it
was suggested on [0] such that it can be properly linked with
Chrome projects.

[0] https://codereview.chromium.org/1818903004/

BUG=500605

Review URL: https://codereview.webrtc.org/1914893005

Cr-Commit-Position: refs/heads/master@{#12515}
2016-04-26 19:55:10 +00:00
cc23b7c1ea Delete unused methods SetStartImage and SetTimeoutImage.
Declared in webrtc::VideoRender, implemented in IncomingVideoStream.
This cl also eliminates some of the few uses of
webrtc::VideoFrame::CopyFrame.

BUG=webrtc:5682

Review URL: https://codereview.webrtc.org/1885323002

Cr-Commit-Position: refs/heads/master@{#12427}
2016-04-19 13:19:47 +00:00
a96b60b3a6 Move frame_callback.h to common_video/include.
BUG=webrtc:4243
R=kjellander@webrtc.org, tommi@webrtc.org

Review URL: https://codereview.webrtc.org/1902543002

Cr-Commit-Position: refs/heads/master@{#12419}
2016-04-19 04:12:57 +00:00
06176e49e2 Added new VideoFrameBuffer methods Data[YUV]() etc.
Eliminate most uses of the old methods.

To continue on this path, once we agree the new methods make sense,
the next step is to rename cricket::VideoFrame::GetVideoFrameBuffer
--> video_frame_buffer, to match the name in webrtc::VideoFrame (if we
think that name is ok?). And then start updating all code to access
planes via the VideoFrameBuffer, and delete corresponding methods in
both cricket::VideoFrame and webrtc::VideoFrame.

BUG=webrtc:5682

Review URL: https://codereview.webrtc.org/1878623002

Cr-Commit-Position: refs/heads/master@{#12407}
2016-04-18 12:34:45 +00:00
47fe34c2bd Introduce an IsMutable method on VideoFrameBuffer.
Unlike HasOneRef, it can be overridden to always return false in
immutable subclasses.

I'm also investigating overiding it in PooledI420Buffer, to directly
inherit I420Buffer but ignore the reference from the pool. Still
unclear if that will work out.

BUG=webrtc:5682

Committed: https://crrev.com/6bd10f2c1ac912cbe5addd880e559d59274c60e6
Cr-Commit-Position: refs/heads/master@{#12365}

R=magjed@webrtc.org, pbos@webrtc.org, perkj@webrtc.org

Review URL: https://codereview.webrtc.org/1881933004 .

Cr-Commit-Position: refs/heads/master@{#12404}
2016-04-18 11:03:11 +00:00
dad23d06aa Revert of Introduce an IsMutable method on VideoFrameBuffer. (patchset #1 id:1 of https://codereview.webrtc.org/1881933004/ )
Reason for revert:
This is breaking all FYI bots.
The new virtual method is not implemented on the Chromium side yet.

Original issue's description:
> Introduce an IsMutable method on VideoFrameBuffer.
>
> Unlike HasOneRef, it can be overridden to always return false in
> immutable subclasses.
>
> I'm also investigating overiding it in PooledI420Buffer, to directly
> inherit I420Buffer but ignore the reference from the pool. Still
> unclear if that will work out.
>
> BUG=webrtc:5682
>
> Committed: https://crrev.com/6bd10f2c1ac912cbe5addd880e559d59274c60e6
> Cr-Commit-Position: refs/heads/master@{#12365}

TBR=magjed@webrtc.org,perkj@webrtc.org,pbos@webrtc.org,nisse@webrtc.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG=webrtc:5682

Review URL: https://codereview.webrtc.org/1885943004

Cr-Commit-Position: refs/heads/master@{#12366}
2016-04-14 16:35:05 +00:00
6bd10f2c1a Introduce an IsMutable method on VideoFrameBuffer.
Unlike HasOneRef, it can be overridden to always return false in
immutable subclasses.

I'm also investigating overiding it in PooledI420Buffer, to directly
inherit I420Buffer but ignore the reference from the pool. Still
unclear if that will work out.

BUG=webrtc:5682

Review URL: https://codereview.webrtc.org/1881933004

Cr-Commit-Position: refs/heads/master@{#12365}
2016-04-14 15:46:44 +00:00
7d06a8cfe4 Add CoreVideoFrameBuffer.
- Makes vt h264 decoder output CoreVideoFrameBuffer
- Makes iOS renderer convert frame buffer if it is not i420

BUG=

Review URL: https://codereview.webrtc.org/1853503003

Cr-Commit-Position: refs/heads/master@{#12224}
2016-04-04 21:10:47 +00:00
7cc9cc0460 New method I420Buffer::Copy.
Needed to replace cricket::VideoFrame::MakeExclusive in chromium.

BUG=webrtc:5682

Review URL: https://codereview.webrtc.org/1822283002

Cr-Commit-Position: refs/heads/master@{#12155}
2016-03-30 06:44:24 +00:00
c891eb479b Replace scoped_ptr with unique_ptr in webrtc/common_video/
BUG=webrtc:5520

Review URL: https://codereview.webrtc.org/1749103003

Cr-Commit-Position: refs/heads/master@{#11838}
2016-03-02 11:41:40 +00:00
739fcb989d Cleanup of webrtc::VideoFrame.
Delete EqualsFrame method, used only by tests. Delete one of the
CreateFrame methods. Drop return value for CreateEmptyFrame, CreateFrame
and CopyFrame.

BUG=webrtc:5426

Committed: https://crrev.com/208019637bfed975f8f13b16d40b90e200763cd6
Cr-Commit-Position: refs/heads/master@{#11783}

R=mflodman@webrtc.org, pbos@webrtc.org, perkj@webrtc.org, pthatcher@webrtc.org

Review URL: https://codereview.webrtc.org/1679323002 .

Cr-Commit-Position: refs/heads/master@{#11811}
2016-02-29 12:11:57 +00:00
54ebfca934 Revert of Cleanup of webrtc::VideoFrame. (patchset #6 id:100001 of https://codereview.webrtc.org/1679323002/ )
Reason for revert:
Breaks downstream compilation. Please make non-breaking API changes for the reland or coordinate fixing downstream code quickly with the sheriff.

Original issue's description:
> Cleanup of webrtc::VideoFrame.
>
> Delete EqualsFrame method, used only by tests. Delete one of the
> CreateFrame methods. Drop return value for CreateEmptyFrame, CreateFrame
> and CopyFrame.
>
> BUG=webrtc:5426
>
> Committed: https://crrev.com/208019637bfed975f8f13b16d40b90e200763cd6
> Cr-Commit-Position: refs/heads/master@{#11783}

TBR=pbos@webrtc.org,perkj@webrtc.org,pthatcher@webrtc.org,mflodman@webrtc.org,marpan@webrtc.org,nisse@webrtc.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG=webrtc:5426

Review URL: https://codereview.webrtc.org/1743613002

Cr-Commit-Position: refs/heads/master@{#11789}
2016-02-26 15:38:57 +00:00
208019637b Cleanup of webrtc::VideoFrame.
Delete EqualsFrame method, used only by tests. Delete one of the
CreateFrame methods. Drop return value for CreateEmptyFrame, CreateFrame
and CopyFrame.

BUG=webrtc:5426

Review URL: https://codereview.webrtc.org/1679323002

Cr-Commit-Position: refs/heads/master@{#11783}
2016-02-26 14:40:47 +00:00
900f97534b H264: Improve FFmpeg decoder performance by using I420BufferPool.
Had to update I420BufferPool to allow zero-initializing buffers.

BUG=chromium:500605, chromium:468365, webrtc:5428

Review URL: https://codereview.webrtc.org/1645543003

Cr-Commit-Position: refs/heads/master@{#11505}
2016-02-05 16:08:39 +00:00
5ad935cb56 Remove mutable from rtc::CriticalSection members.
rtc::CriticalSection is now lockable from const methods and no longer
need to remain mutable.

BUG=
R=tommi@webrtc.org

Review URL: https://codereview.webrtc.org/1613643004

Cr-Commit-Position: refs/heads/master@{#11367}
2016-01-25 11:52:53 +00:00
84df580d52 Switch to rtc::CriticalSection in IncomingVideoStream and remove one lock.
BUG=

Review URL: https://codereview.webrtc.org/1608743005

Cr-Commit-Position: refs/heads/master@{#11318}
2016-01-20 12:38:00 +00:00
8c38e8b9b9 Clean up PlatformThread.
* Move PlatformThread to rtc::.
* Remove ::CreateThread factory method.
* Make non-scoped_ptr from a lot of invocations.
* Make Start/Stop void.
* Remove rtc::Thread priorities, which were unused and would collide.
* Add ::IsRunning() to PlatformThread.

BUG=
R=tommi@webrtc.org

Review URL: https://codereview.webrtc.org/1476453002 .

Cr-Commit-Position: refs/heads/master@{#10812}
2015-11-26 16:45:57 +00:00
444682acf9 Remove frame time scheduing in IncomingVideoStream
This is part of the project that makes RTC rendering more
smooth. We've already finished the developement of the
frame selection algorithm in WebMediaPlayerMS, where we
managed a frame pool, and based on the vsync interval, we
actively select the best frame to render in order to
maximize the rendering smoothness.

Thus the frame timeline control in IncomingVideoStream is
no longer needed, because with sophisticated frame
selection algorithm in WebMediaPlayerMS, the time control
in IncomingVideoStream will do nothing but add some extra
delay.

BUG=514873

Review URL: https://codereview.webrtc.org/1419673014

Cr-Commit-Position: refs/heads/master@{#10781}
2015-11-25 02:08:03 +00:00
12411ef40e Move ThreadWrapper to ProcessThread in base.
Also removes all virtual methods. Permits using a thread from
rtc_base_approved (namely event tracing).

BUG=webrtc:5158
R=tommi@webrtc.org

Review URL: https://codereview.webrtc.org/1469013002

Cr-Commit-Position: refs/heads/master@{#10760}
2015-11-23 22:48:01 +00:00
6f8ce060a2 common_video: rename interface -> include
To avoid breaking downstream, the "interface" directories were copied
into a new "common_video/include" dir. The old headers got pragma
warnings added about deprecation (a very short deprecation since I plan
to remove them as soon downstream is updated).
The header guards are also identical to avoid mixing them up in the transition.

BUG=webrtc:5095
TESTED=Passing compile-trybots with --clobber flag:
git cl try --clobber --bot=win_compile_rel --bot=linux_compile_rel --bot=android_compile_rel --bot=mac_compile_rel --bot=ios_rel --bot=linux_gn_rel --bot=win_x64_gn_rel --bot=mac_x64_gn_rel --bot=android_gn_rel -m tryserver.webrtc

Review URL: https://codereview.webrtc.org/1418913006

Cr-Commit-Position: refs/heads/master@{#10659}
2015-11-16 21:52:31 +00:00