Commit Graph

158 Commits

Author SHA1 Message Date
fd5a20fd68 New jitter buffer experiment.
BUG=webrtc:5514

Review-Url: https://codereview.webrtc.org/2480293002
Cr-Commit-Position: refs/heads/master@{#15077}
2016-11-15 08:58:06 +00:00
b4bc65b4e9 Fix circular dependency between call and video receive stream.
BUG=webrtc:4243

Review-Url: https://codereview.webrtc.org/2469293003
Cr-Commit-Position: refs/heads/master@{#14899}
2016-11-02 17:10:26 +00:00
de9e5fffa2 Add stats for frequency offset when converting RTP timestamp to NTP time.
- Add histogram: "WebRTC.Video.RtpToNtpFreqOffsetInKhz"

  The absolute value of the difference between the estimated frequency during RTP timestamp to NTP time conversion and the actual value (i.e. 90 kHz) is measured per received video frame. The max offset during 40 second intervals is stored. The average of these stored offsets per received video stream is recorded when a stream is removed.

Updated rtp_to_ntp.cc:
- Add validation for only inserting newer RTCP sender reports to the rtcp list.

- Move calculation of frequency/offset (from RTP/NTP timestamp pairs) to UpdateRtcpList. Calculated when a new RTCP SR in inserted (and not in RtpToNtpMs per packet).

BUG=webrtc:6579

Review-Url: https://codereview.webrtc.org/2385763002
Cr-Commit-Position: refs/heads/master@{#14891}
2016-11-02 14:14:10 +00:00
hta
257dc39841 Refactoring: Hide VideoCodec.codecSpecific as "private"
This refactoring allows runtime checks that functions that access
codec specific information are using the correct union member.
The API also allows replacing the union with another implementation
without changes at calling sites.

BUG=webrtc:6603

Review-Url: https://codereview.webrtc.org/2001533003
Cr-Commit-Position: refs/heads/master@{#14775}
2016-10-25 16:05:15 +00:00
4e52386339 Reland of Add path for recovered packets from internal::Call to RtpStreamReceiver. (patchset #1 id:1 of https://codereview.webrtc.org/2427733002/ )
Reason for revert:
Flaky test has been fixed.

Original issue's description:
> Revert of Add path for recovered packets from internal::Call to RtpStreamReceiver. (patchset #2 id:60001 of https://codereview.webrtc.org/2390823009/ )
>
> Reason for revert:
> Speculative revert as it may be the cause of the DrMemory test failure:
> https://build.chromium.org/p/client.webrtc/builders/Win%20DrMemory%20Full/builds/5115
>
> Original issue's description:
> > Add path for recovered packets from internal::Call to RtpStreamReceiver.
> >
> > When the FlexfecReceiver recovers media packets, it inserts these into
> > internal::Call, which then distributes them to the appropriate
> > VideoReceiveStream/RtpStreamReceiver.
> >
> > BUG=webrtc:5654
> >
> > Committed: https://crrev.com/9c4b4b47f4325b48e1856566a30983f9e4e30dd0
> > Cr-Commit-Position: refs/heads/master@{#14642}
>
> TBR=stefan@webrtc.org,brandtr@webrtc.org
> # Skipping CQ checks because original CL landed less than 1 days ago.
> NOPRESUBMIT=true
> NOTREECHECKS=true
> NOTRY=true
> BUG=webrtc:5654
>
> Committed: https://crrev.com/862d74d0176fa762b3c96cf20bd36f27e7001a47
> Cr-Commit-Position: refs/heads/master@{#14652}

TBR=stefan@webrtc.org,honghaiz@webrtc.org
# Not skipping CQ checks because original CL landed more than 1 days ago.
BUG=webrtc:5654

Review-Url: https://codereview.webrtc.org/2428303004
Cr-Commit-Position: refs/heads/master@{#14677}
2016-10-19 06:50:53 +00:00
862d74d017 Revert of Add path for recovered packets from internal::Call to RtpStreamReceiver. (patchset #2 id:60001 of https://codereview.webrtc.org/2390823009/ )
Reason for revert:
Speculative revert as it may be the cause of the DrMemory test failure:
https://build.chromium.org/p/client.webrtc/builders/Win%20DrMemory%20Full/builds/5115

Original issue's description:
> Add path for recovered packets from internal::Call to RtpStreamReceiver.
>
> When the FlexfecReceiver recovers media packets, it inserts these into
> internal::Call, which then distributes them to the appropriate
> VideoReceiveStream/RtpStreamReceiver.
>
> BUG=webrtc:5654
>
> Committed: https://crrev.com/9c4b4b47f4325b48e1856566a30983f9e4e30dd0
> Cr-Commit-Position: refs/heads/master@{#14642}

TBR=stefan@webrtc.org,brandtr@webrtc.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG=webrtc:5654

Review-Url: https://codereview.webrtc.org/2427733002
Cr-Commit-Position: refs/heads/master@{#14652}
2016-10-17 16:42:38 +00:00
9c4b4b47f4 Add path for recovered packets from internal::Call to RtpStreamReceiver.
When the FlexfecReceiver recovers media packets, it inserts these into
internal::Call, which then distributes them to the appropriate
VideoReceiveStream/RtpStreamReceiver.

BUG=webrtc:5654

Review-Url: https://codereview.webrtc.org/2390823009
Cr-Commit-Position: refs/heads/master@{#14642}
2016-10-16 21:11:00 +00:00
113bdcadf3 Make sure VideoReceiveStream can be restarted
After calling Start(), doing a Stop() then Start() sequence should bring
the stream back to the original state.

BUG=webrtc:6501

Review-Url: https://codereview.webrtc.org/2407163002
Cr-Commit-Position: refs/heads/master@{#14597}
2016-10-11 10:10:13 +00:00
0d348d69e6 Avoid race in VideoReceiveStream shutdown
The decoder implementation may have its own thread for asynchrouns
callbacks. We need to stop it (by releasing the decoder) when stopping
the decoder thread, othweise it may call incoming_video_stream_ after
it has been destroyed.

BUG=webrtc:6501

Review-Url: https://codereview.webrtc.org/2402663003
Cr-Commit-Position: refs/heads/master@{#14577}
2016-10-07 15:28:42 +00:00
b5f2c3fbe9 Rename FecConfig to UlpfecConfig in config.h.
Also rename some related minor methods. No functional changes
are intended/expected.

BUG=webrtc:5654

Review-Url: https://codereview.webrtc.org/2391963002
Cr-Commit-Position: refs/heads/master@{#14513}
2016-10-05 06:28:43 +00:00
55d932b331 Add logging statements to places where the frame might be dropped in WebRTC pipeline.
BUG=b/31645554

Review-Url: https://codereview.webrtc.org/2361803003
Cr-Commit-Position: refs/heads/master@{#14457}
2016-09-30 13:19:12 +00:00
e75f204b06 Expose Ivf logging through the native API
BUG=webrtc:6300

Review-Url: https://codereview.webrtc.org/2303273002
Cr-Commit-Position: refs/heads/master@{#14419}
2016-09-28 13:19:53 +00:00
1490f7aa55 Add histogram for end-to-end delay:
"WebRTC.Video.EndToEndDelayInMs"

Make capture time in local timebase available for decoded VP9 video frames (propagate ntp_time_ms from EncodedImage to decoded VideoFrame).

BUG=webrtc:6409

Review-Url: https://codereview.webrtc.org/1905563002
Cr-Commit-Position: refs/heads/master@{#14367}
2016-09-23 09:09:59 +00:00
2e5cfcd6c2 Add periodic logging of video stats.
Add ToString method to: Call::Stats, VideoSendStream::Stats, VideoReceiveStream::Stats and log stats periodically (every 10 seconds).

BUG=

Review-Url: https://codereview.webrtc.org/2133073002
Cr-Commit-Position: refs/heads/master@{#13727}
2016-08-11 15:41:26 +00:00
3859c89b2e Add decoder-specific settings with proper lifetime.
Utilize these settings for h264 sprop-parameter-sets.

BUG=webrtc:5948

Review-Url: https://codereview.webrtc.org/2185953002
Cr-Commit-Position: refs/heads/master@{#13656}
2016-08-05 16:19:33 +00:00
4cd2790f17 Move RTP for synchroninzation and rename classes, files and variables.
This CL removes (almost) the last RTP references in VideoReceiveStream.
There are still references to RTPFragmentationHeader and SSRCs, which
will be dealt with later.

There are also new GUARDED_BY and thred checker added to the
synchronization class.

When there are othre transports than RTP, there will instead be an
interface + inheritance for RtpStreamReceiver and
RtpStreamSynchronizattion in VideoReceiveStream. This work will be done
when we actually know how we want to make thee transport interface.

BUG=webrtc:5838

Review-Url: https://codereview.webrtc.org/2216533002
Cr-Commit-Position: refs/heads/master@{#13655}
2016-08-05 13:28:50 +00:00
525df3ffd1 Add EncodedImageCallback::OnEncodedImage().
OnEncodedImage() is going to replace Encoded(), which is deprecated now.
The new OnEncodedImage() returns Result struct that contains frame_id,
which tells the encoder RTP timestamp for the frame.

BUG=chromium:621691
R=niklas.enbom@webrtc.org, sprang@webrtc.org, stefan@webrtc.org

Review URL: https://codereview.webrtc.org/2089773002 .

Committed: https://crrev.com/4c7f4cd2ef76821edca6d773d733a924b0bedd25
Committed: https://crrev.com/ad34dbe934d47f88011045671b4aea00dbd5a795
Cr-Original-Original-Commit-Position: refs/heads/master@{#13613}
Cr-Original-Commit-Position: refs/heads/master@{#13615}
Cr-Commit-Position: refs/heads/master@{#13617}
2016-08-03 00:46:47 +00:00
51db4dd1bd Revert of Add EncodedImageCallback::OnEncodedImage(). (patchset #14 id:300001 of https://codereview.chromium.org/2089773002/ )
Reason for revert:
broke browser_tests

Original issue's description:
> Add EncodedImageCallback::OnEncodedImage().
>
> OnEncodedImage() is going to replace Encoded(), which is deprecated now.
> The new OnEncodedImage() returns Result struct that contains frame_id,
> which tells the encoder RTP timestamp for the frame.
>
> BUG=chromium:621691
> R=niklas.enbom@webrtc.org, sprang@webrtc.org, stefan@webrtc.org
>
> Committed: https://crrev.com/4c7f4cd2ef76821edca6d773d733a924b0bedd25
> Committed: https://crrev.com/ad34dbe934d47f88011045671b4aea00dbd5a795
> Cr-Original-Commit-Position: refs/heads/master@{#13613}
> Cr-Commit-Position: refs/heads/master@{#13615}

TBR=pbos@webrtc.org,mflodman@webrtc.org,sprang@webrtc.org,stefan@webrtc.org,niklas.enbom@webrtc.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG=chromium:621691

Review-Url: https://codereview.webrtc.org/2203233002
Cr-Commit-Position: refs/heads/master@{#13616}
2016-08-03 00:33:47 +00:00
4c7f4cd2ef Add EncodedImageCallback::OnEncodedImage().
OnEncodedImage() is going to replace Encoded(), which is deprecated now.
The new OnEncodedImage() returns Result struct that contains frame_id,
which tells the encoder RTP timestamp for the frame.

BUG=chromium:621691
R=niklas.enbom@webrtc.org, sprang@webrtc.org, stefan@webrtc.org

Review URL: https://codereview.webrtc.org/2089773002 .

Committed: https://crrev.com/ad34dbe934d47f88011045671b4aea00dbd5a795
Cr-Original-Commit-Position: refs/heads/master@{#13613}
Cr-Commit-Position: refs/heads/master@{#13615}
2016-08-02 22:14:51 +00:00
ac4dc2cefe Revert of Add EncodedImageCallback::OnEncodedImage(). (patchset #13 id:280001 of https://codereview.webrtc.org/2089773002/ )
Reason for revert:
broke internal tests

Original issue's description:
> Add EncodedImageCallback::OnEncodedImage().
>
> OnEncodedImage() is going to replace Encoded(), which is deprecated now.
> The new OnEncodedImage() returns Result struct that contains frame_id,
> which tells the encoder RTP timestamp for the frame.
>
> BUG=chromium:621691
> R=niklas.enbom@webrtc.org, sprang@webrtc.org, stefan@webrtc.org
>
> Committed: https://crrev.com/ad34dbe934d47f88011045671b4aea00dbd5a795
> Cr-Commit-Position: refs/heads/master@{#13613}

TBR=pbos@webrtc.org,mflodman@webrtc.org,sprang@webrtc.org,stefan@webrtc.org,niklas.enbom@webrtc.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG=chromium:621691

Review-Url: https://codereview.webrtc.org/2206743002
Cr-Commit-Position: refs/heads/master@{#13614}
2016-08-02 21:33:21 +00:00
ad34dbe934 Add EncodedImageCallback::OnEncodedImage().
OnEncodedImage() is going to replace Encoded(), which is deprecated now.
The new OnEncodedImage() returns Result struct that contains frame_id,
which tells the encoder RTP timestamp for the frame.

BUG=chromium:621691
R=niklas.enbom@webrtc.org, sprang@webrtc.org, stefan@webrtc.org

Review URL: https://codereview.webrtc.org/2089773002 .

Cr-Commit-Position: refs/heads/master@{#13613}
2016-08-02 20:44:25 +00:00
737336d37a Add NACK rate throttling for audio channels.
Not really used for audio today (already in place for video), but should
still function anyway.

BUG=
R=henrika@webrtc.org, minyue@webrtc.org, stefan@webrtc.org

Review URL: https://codereview.webrtc.org/2181383002 .

Cr-Commit-Position: refs/heads/master@{#13571}
2016-07-29 10:59:49 +00:00
22691e0d32 Avoid race in VideoReceiveStream shutdown
The decode thread should be stopped before triggering shutdown of the
video receiver, so that the decoder doesn't try to insert a new frame
while the jitter buffer is being shut down.

BUG=webrtc:6102

Review-Url: https://codereview.webrtc.org/2146883002
Cr-Commit-Position: refs/heads/master@{#13467}
2016-07-13 17:57:20 +00:00
2e82f3821f Reland of Split IncomingVideoStream into two implementations, with smoothing and without. (patchset #1 id:1 of https://codereview.webrtc.org/2084873002/ )
Reason for revert:
Reverting the revert.  This change is not related to the failure on the Windows FYI bots.  The cause of the failure has been reverted in Chromium:
https://codereview.chromium.org/2081653004/

Original issue's description:
> Revert of Split IncomingVideoStream into two implementations, with smoothing and without. (patchset #5 id:340001 of https://codereview.webrtc.org/2078873002/ )
>
> Reason for revert:
> Breaks chromium.webrtc.fyi
>
> https://uberchromegw.corp.google.com/i/chromium.webrtc.fyi/builders/Win7%20Tester/builds/4719
> https://uberchromegw.corp.google.com/i/chromium.webrtc.fyi/builders/Win10%20Tester/builds/3120
>
> Original issue's description:
> > Reland of IncomingVideoStream refactoring.
> > This reland does not contain the non-smoothing part of the original implementation.  Instead, when smoothing is turned off, frame callbacks run on the decoder thread, as they did before.  This code path is used in Chrome.  As far as Chrome goes, the difference now is that there won't be an instance of IncomingVideoStream in between the decoder and the callback (i.e. fewer locks).  Other than that, no change for Chrome.
> >
> > Original issue's description (with non-smoothing references removed):
> >
> > Split IncomingVideoStream into two implementations, with smoothing and without.
> >
> > * Added TODOs and documentation for VideoReceiveStream::OnFrame, where we today grab 6 locks.
> >
> > * Removed the Start/Stop methods from the IncomingVideoStream implementations.  Now, when an instance is created, it should be considered to be "running" and when it is deleted, it's "not running".  This saves on resources and also reduces the amount of locking required and I could remove one critical section altogether.
> >
> > * Changed the VideoStreamDecoder class to not depend on IncomingVideoStream but rather use the generic rtc::VideoSinkInterface<VideoFrame> interface.  This means that any implementation of that interface can be used and the decoder can be made to  just use the 'renderer' from the config.  Once we do that, we can decouple the IncomingVideoStream implementations from the decoder and VideoReceiveStream implementations and leave it up to the application for how to do smoothing.  The app can choose to use the Incoming* classes or roll its own (which may be preferable since applications often have their own scheduling mechanisms).
> >
> > * The lifetime of the VideoStreamDecoder instance is now bound to Start/Stop in VideoReceiveStream and not all of the lifetime of VideoReceiveStream.
> >
> > * Fixed VideoStreamDecoder to unregister callbacks in the dtor that were registered in the ctor. (this was open to a use-after-free regression)
> >
> > * Delay and callback pointers are now passed via the ctors to the IncomingVideoStream classes.  The thread primitives in the IncomingVideoStream classes are also constructed/destructed at the same time as the owning object, which allowed me to remove one more lock.
> >
> > * Removed code in the VideoStreamDecoder that could overwrite the VideoReceiveStream render delay with a fixed value of 10ms on construction.  This wasn't a problem with the previous implementation (it would be now though) but seemed to me like the wrong place to be setting that value.
> >
> > * Made the render delay value in VideoRenderFrames, const.
> >
> > BUG=chromium:620232
> > R=mflodman@webrtc.org, nisse@webrtc.org
> >
> > Committed: https://crrev.com/884c336c345d988974c2a69cea402b0fb8b07a63
> > Cr-Commit-Position: refs/heads/master@{#13219}
>
> TBR=nisse@webrtc.org,philipel@webrtc.org,mflodman@webrtc.org,tommi@webrtc.org
> # Skipping CQ checks because original CL landed less than 1 days ago.
> NOPRESUBMIT=true
> NOTREECHECKS=true
> NOTRY=true
> BUG=chromium:620232
>
> Committed: https://crrev.com/a536bfe70de38fe877245317a7f0b00bcf69cbd0
> Cr-Commit-Position: refs/heads/master@{#13229}

TBR=nisse@webrtc.org,philipel@webrtc.org,mflodman@webrtc.org,sakal@webrtc.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG=chromium:620232

Review-Url: https://codereview.webrtc.org/2089613002
Cr-Commit-Position: refs/heads/master@{#13230}
2016-06-21 07:26:48 +00:00
a536bfe70d Revert of Split IncomingVideoStream into two implementations, with smoothing and without. (patchset #5 id:340001 of https://codereview.webrtc.org/2078873002/ )
Reason for revert:
Breaks chromium.webrtc.fyi

https://uberchromegw.corp.google.com/i/chromium.webrtc.fyi/builders/Win7%20Tester/builds/4719
https://uberchromegw.corp.google.com/i/chromium.webrtc.fyi/builders/Win10%20Tester/builds/3120

Original issue's description:
> Reland of IncomingVideoStream refactoring.
> This reland does not contain the non-smoothing part of the original implementation.  Instead, when smoothing is turned off, frame callbacks run on the decoder thread, as they did before.  This code path is used in Chrome.  As far as Chrome goes, the difference now is that there won't be an instance of IncomingVideoStream in between the decoder and the callback (i.e. fewer locks).  Other than that, no change for Chrome.
>
> Original issue's description (with non-smoothing references removed):
>
> Split IncomingVideoStream into two implementations, with smoothing and without.
>
> * Added TODOs and documentation for VideoReceiveStream::OnFrame, where we today grab 6 locks.
>
> * Removed the Start/Stop methods from the IncomingVideoStream implementations.  Now, when an instance is created, it should be considered to be "running" and when it is deleted, it's "not running".  This saves on resources and also reduces the amount of locking required and I could remove one critical section altogether.
>
> * Changed the VideoStreamDecoder class to not depend on IncomingVideoStream but rather use the generic rtc::VideoSinkInterface<VideoFrame> interface.  This means that any implementation of that interface can be used and the decoder can be made to  just use the 'renderer' from the config.  Once we do that, we can decouple the IncomingVideoStream implementations from the decoder and VideoReceiveStream implementations and leave it up to the application for how to do smoothing.  The app can choose to use the Incoming* classes or roll its own (which may be preferable since applications often have their own scheduling mechanisms).
>
> * The lifetime of the VideoStreamDecoder instance is now bound to Start/Stop in VideoReceiveStream and not all of the lifetime of VideoReceiveStream.
>
> * Fixed VideoStreamDecoder to unregister callbacks in the dtor that were registered in the ctor. (this was open to a use-after-free regression)
>
> * Delay and callback pointers are now passed via the ctors to the IncomingVideoStream classes.  The thread primitives in the IncomingVideoStream classes are also constructed/destructed at the same time as the owning object, which allowed me to remove one more lock.
>
> * Removed code in the VideoStreamDecoder that could overwrite the VideoReceiveStream render delay with a fixed value of 10ms on construction.  This wasn't a problem with the previous implementation (it would be now though) but seemed to me like the wrong place to be setting that value.
>
> * Made the render delay value in VideoRenderFrames, const.
>
> BUG=chromium:620232
> R=mflodman@webrtc.org, nisse@webrtc.org
>
> Committed: https://crrev.com/884c336c345d988974c2a69cea402b0fb8b07a63
> Cr-Commit-Position: refs/heads/master@{#13219}

TBR=nisse@webrtc.org,philipel@webrtc.org,mflodman@webrtc.org,tommi@webrtc.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG=chromium:620232

Review-Url: https://codereview.webrtc.org/2084873002
Cr-Commit-Position: refs/heads/master@{#13229}
2016-06-21 07:08:58 +00:00
884c336c34 Reland of IncomingVideoStream refactoring.
This reland does not contain the non-smoothing part of the original implementation.  Instead, when smoothing is turned off, frame callbacks run on the decoder thread, as they did before.  This code path is used in Chrome.  As far as Chrome goes, the difference now is that there won't be an instance of IncomingVideoStream in between the decoder and the callback (i.e. fewer locks).  Other than that, no change for Chrome.

Original issue's description (with non-smoothing references removed):

Split IncomingVideoStream into two implementations, with smoothing and without.

* Added TODOs and documentation for VideoReceiveStream::OnFrame, where we today grab 6 locks.

* Removed the Start/Stop methods from the IncomingVideoStream implementations.  Now, when an instance is created, it should be considered to be "running" and when it is deleted, it's "not running".  This saves on resources and also reduces the amount of locking required and I could remove one critical section altogether.

* Changed the VideoStreamDecoder class to not depend on IncomingVideoStream but rather use the generic rtc::VideoSinkInterface<VideoFrame> interface.  This means that any implementation of that interface can be used and the decoder can be made to  just use the 'renderer' from the config.  Once we do that, we can decouple the IncomingVideoStream implementations from the decoder and VideoReceiveStream implementations and leave it up to the application for how to do smoothing.  The app can choose to use the Incoming* classes or roll its own (which may be preferable since applications often have their own scheduling mechanisms).

* The lifetime of the VideoStreamDecoder instance is now bound to Start/Stop in VideoReceiveStream and not all of the lifetime of VideoReceiveStream.

* Fixed VideoStreamDecoder to unregister callbacks in the dtor that were registered in the ctor. (this was open to a use-after-free regression)

* Delay and callback pointers are now passed via the ctors to the IncomingVideoStream classes.  The thread primitives in the IncomingVideoStream classes are also constructed/destructed at the same time as the owning object, which allowed me to remove one more lock.

* Removed code in the VideoStreamDecoder that could overwrite the VideoReceiveStream render delay with a fixed value of 10ms on construction.  This wasn't a problem with the previous implementation (it would be now though) but seemed to me like the wrong place to be setting that value.

* Made the render delay value in VideoRenderFrames, const.

BUG=chromium:620232
R=mflodman@webrtc.org, nisse@webrtc.org

Review URL: https://codereview.webrtc.org/2078873002 .

Cr-Commit-Position: refs/heads/master@{#13219}
2016-06-20 17:43:10 +00:00
8e8222d0d2 Revert of Split IncomingVideoStream into two implementations, with smoothing and without. (patchset #4 id:290001 of https://codereview.webrtc.org/2071473002/ )
Reason for revert:
Reverting again.  The perf regression does not seem to be related to dropping frames.

Original issue's description:
> Reland of Split IncomingVideoStream into two implementations, with smoothing and without.
>
> Original issue's description:
>
> Split IncomingVideoStream into two implementations, with smoothing and without.
>
> This CL fixes an issue with the non-smoothing implementation where frames were delivered on the decoder thread.  No-smoothing is now done in a separate class that uses a TaskQueue.  The implementation may drop frames if the renderer doesn't keep up and it doesn't block the decoder thread.
>
> Further work done:
>
> * I added TODOs and documentation for VideoReceiveStream::OnFrame, where we today grab 5 locks.
>
> * I removed the Start/Stop methods from the IncomingVideoStream implementations.  Now, when an instance is created, it should be considered to be "running" and when it is deleted, it's "not running".  This saves on resources and also reduces the amount of locking required and I could remove one critical section altogether.
>
> * I changed the VideoStreamDecoder class to not depend on IncomingVideoStream but rather use the generic rtc::VideoSinkInterface<VideoFrame> interface.  This means that any implementation of that interface can be used and the decoder can be made to  just use the 'renderer' from the config.  Once we do that, we can decouple the IncomingVideoStream implementations from the decoder and VideoReceiveStream implementations and leave it up to the application for how to do smoothing.  The app can choose to use the Incoming* classes or roll its own (which may be preferable since applications often have their own scheduling mechanisms).
>
> * The non-smoothing IncomingVideoStream implementation currently allows only 1 outstanding pending frame.  If we exceed that, the current frame won't be delivered to the renderer and instead we deliver the next one (since when this happens, the renderer is falling behind).
>
> * The lifetime of the VideoStreamDecoder instance is now bound to Start/Stop in VideoReceiveStream and not all of the lifetime of VideoReceiveStream.
>
> * Fixed VideoStreamDecoder to unregister callbacks in the dtor that were registered in the ctor. (this was open to a use-after-free regression)
>
> * Delay and callback pointers are now passed via the ctors to the IncomingVideoStream classes.  The thread primitives in the IncomingVideoStream classes are also constructed/destructed at the same time as the owning object, which allowed me to remove one more lock.
>
> * Removed code in the VideoStreamDecoder that could overwrite the VideoReceiveStream render delay with a fixed value of 10ms on construction.  This wasn't a problem with the previous implementation (it would be now though) but seemed to me like the wrong place to be setting that value.
>
> * Made the render delay value in VideoRenderFrames, const.
>
> BUG=chromium:620232
> TBR=mflodman
>
> Committed: https://crrev.com/e03f8787377bbc03a4e00184bb14b7561b108cbb
> Cr-Commit-Position: refs/heads/master@{#13175}

TBR=mflodman@webrtc.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG=chromium:620232

Review-Url: https://codereview.webrtc.org/2071093002
Cr-Commit-Position: refs/heads/master@{#13176}
2016-06-16 22:44:11 +00:00
e03f878737 Reland of Split IncomingVideoStream into two implementations, with smoothing and without.
Original issue's description:

Split IncomingVideoStream into two implementations, with smoothing and without.

This CL fixes an issue with the non-smoothing implementation where frames were delivered on the decoder thread.  No-smoothing is now done in a separate class that uses a TaskQueue.  The implementation may drop frames if the renderer doesn't keep up and it doesn't block the decoder thread.

Further work done:

* I added TODOs and documentation for VideoReceiveStream::OnFrame, where we today grab 5 locks.

* I removed the Start/Stop methods from the IncomingVideoStream implementations.  Now, when an instance is created, it should be considered to be "running" and when it is deleted, it's "not running".  This saves on resources and also reduces the amount of locking required and I could remove one critical section altogether.

* I changed the VideoStreamDecoder class to not depend on IncomingVideoStream but rather use the generic rtc::VideoSinkInterface<VideoFrame> interface.  This means that any implementation of that interface can be used and the decoder can be made to  just use the 'renderer' from the config.  Once we do that, we can decouple the IncomingVideoStream implementations from the decoder and VideoReceiveStream implementations and leave it up to the application for how to do smoothing.  The app can choose to use the Incoming* classes or roll its own (which may be preferable since applications often have their own scheduling mechanisms).

* The non-smoothing IncomingVideoStream implementation currently allows only 1 outstanding pending frame.  If we exceed that, the current frame won't be delivered to the renderer and instead we deliver the next one (since when this happens, the renderer is falling behind).

* The lifetime of the VideoStreamDecoder instance is now bound to Start/Stop in VideoReceiveStream and not all of the lifetime of VideoReceiveStream.

* Fixed VideoStreamDecoder to unregister callbacks in the dtor that were registered in the ctor. (this was open to a use-after-free regression)

* Delay and callback pointers are now passed via the ctors to the IncomingVideoStream classes.  The thread primitives in the IncomingVideoStream classes are also constructed/destructed at the same time as the owning object, which allowed me to remove one more lock.

* Removed code in the VideoStreamDecoder that could overwrite the VideoReceiveStream render delay with a fixed value of 10ms on construction.  This wasn't a problem with the previous implementation (it would be now though) but seemed to me like the wrong place to be setting that value.

* Made the render delay value in VideoRenderFrames, const.

BUG=chromium:620232
TBR=mflodman

Review-Url: https://codereview.webrtc.org/2071473002
Cr-Commit-Position: refs/heads/master@{#13175}
2016-06-16 20:29:12 +00:00
17c3cddf9d Revert of Split IncomingVideoStream into two implementations, with smoothing and without. (patchset #23 id:430001 of https://codereview.webrtc.org/2035173002/ )
Reason for revert:
Reverting while we track down the issue on the Win10 bot.

Original issue's description:
> Split IncomingVideoStream into two implementations, with smoothing and without.
>
> This CL fixes an issue with the non-smoothing implementation where frames were delivered on the decoder thread.  No-smoothing is now done in a separate class that uses a TaskQueue.  The implementation may drop frames if the renderer doesn't keep up and it doesn't block the decoder thread.
>
> Further work done:
>
> * I added TODOs and documentation for VideoReceiveStream::OnFrame, where we today grab 5 locks.
>
> * I removed the Start/Stop methods from the IncomingVideoStream implementations.  Now, when an instance is created, it should be considered to be "running" and when it is deleted, it's "not running".  This saves on resources and also reduces the amount of locking required and I could remove one critical section altogether.
>
> * I changed the VideoStreamDecoder class to not depend on IncomingVideoStream but rather use the generic rtc::VideoSinkInterface<VideoFrame> interface.  This means that any implementation of that interface can be used and the decoder can be made to  just use the 'renderer' from the config.  Once we do that, we can decouple the IncomingVideoStream implementations from the decoder and VideoReceiveStream implementations and leave it up to the application for how to do smoothing.  The app can choose to use the Incoming* classes or roll its own (which may be preferable since applications often have their own scheduling mechanisms).
>
> * The non-smoothing IncomingVideoStream implementation currently allows only 1 outstanding pending frame.  If we exceed that, the current frame won't be delivered to the renderer and instead we deliver the next one (since when this happens, the renderer is falling behind).
>
> * The lifetime of the VideoStreamDecoder instance is now bound to Start/Stop in VideoReceiveStream and not all of the lifetime of VideoReceiveStream.
>
> * Fixed VideoStreamDecoder to unregister callbacks in the dtor that were registered in the ctor. (this was open to a use-after-free regression)
>
> * Delay and callback pointers are now passed via the ctors to the IncomingVideoStream classes.  The thread primitives in the IncomingVideoStream classes are also constructed/destructed at the same time as the owning object, which allowed me to remove one more lock.
>
> * Removed code in the VideoStreamDecoder that could overwrite the VideoReceiveStream render delay with a fixed value of 10ms on construction.  This wasn't a problem with the previous implementation (it would be now though) but seemed to me like the wrong place to be setting that value.
>
> * Made the render delay value in VideoRenderFrames, const.
>
> BUG=
>
> Committed: https://crrev.com/1c7eef652b0aa22d8ebb0bfe2b547094a794be22
> Cr-Commit-Position: refs/heads/master@{#13129}

TBR=mflodman@webrtc.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG=

Review-Url: https://codereview.webrtc.org/2061363002
Cr-Commit-Position: refs/heads/master@{#13146}
2016-06-14 23:04:48 +00:00
37ad337848 Remove EncodedFrameCallbackAdapter.
EncodedFrameCallbackAdapter was used VideoSendStream and
VideoReceiveStream, but there is no reason to have it as these classes
can call EncodedFrameObserver directly.

Review-Url: https://codereview.webrtc.org/2068463004
Cr-Commit-Position: refs/heads/master@{#13145}
2016-06-14 22:29:45 +00:00
1c7eef652b Split IncomingVideoStream into two implementations, with smoothing and without.
This CL fixes an issue with the non-smoothing implementation where frames were delivered on the decoder thread.  No-smoothing is now done in a separate class that uses a TaskQueue.  The implementation may drop frames if the renderer doesn't keep up and it doesn't block the decoder thread.

Further work done:

* I added TODOs and documentation for VideoReceiveStream::OnFrame, where we today grab 5 locks.

* I removed the Start/Stop methods from the IncomingVideoStream implementations.  Now, when an instance is created, it should be considered to be "running" and when it is deleted, it's "not running".  This saves on resources and also reduces the amount of locking required and I could remove one critical section altogether.

* I changed the VideoStreamDecoder class to not depend on IncomingVideoStream but rather use the generic rtc::VideoSinkInterface<VideoFrame> interface.  This means that any implementation of that interface can be used and the decoder can be made to  just use the 'renderer' from the config.  Once we do that, we can decouple the IncomingVideoStream implementations from the decoder and VideoReceiveStream implementations and leave it up to the application for how to do smoothing.  The app can choose to use the Incoming* classes or roll its own (which may be preferable since applications often have their own scheduling mechanisms).

* The non-smoothing IncomingVideoStream implementation currently allows only 1 outstanding pending frame.  If we exceed that, the current frame won't be delivered to the renderer and instead we deliver the next one (since when this happens, the renderer is falling behind).

* The lifetime of the VideoStreamDecoder instance is now bound to Start/Stop in VideoReceiveStream and not all of the lifetime of VideoReceiveStream.

* Fixed VideoStreamDecoder to unregister callbacks in the dtor that were registered in the ctor. (this was open to a use-after-free regression)

* Delay and callback pointers are now passed via the ctors to the IncomingVideoStream classes.  The thread primitives in the IncomingVideoStream classes are also constructed/destructed at the same time as the owning object, which allowed me to remove one more lock.

* Removed code in the VideoStreamDecoder that could overwrite the VideoReceiveStream render delay with a fixed value of 10ms on construction.  This wasn't a problem with the previous implementation (it would be now though) but seemed to me like the wrong place to be setting that value.

* Made the render delay value in VideoRenderFrames, const.

BUG=

Review-Url: https://codereview.webrtc.org/2035173002
Cr-Commit-Position: refs/heads/master@{#13129}
2016-06-14 11:38:43 +00:00
733b5478dd Movable support for VideoReceiveStream::Config and avoid copies.
Instead of the default copy constructor, the Copy() method has to be used.  In this CL, the number of copies has been reduced significantly in production code. One case in the video engine remains, where we need to restart a video stream.  Even in that case, I'm sure we could avoid it, but for this particular CL, I decided against it to keep things simple (and it's also an edge case).  Most importantly, creating copies is made harder and the interface encourages ownership transfers.

R=mflodman@webrtc.org, pbos@webrtc.org

Review URL: https://codereview.webrtc.org/2042603002 .

Cr-Commit-Position: refs/heads/master@{#13102}
2016-06-10 15:58:12 +00:00
bd3380ff7e Make VideoReceiveStream not inherit from I420FrameCallback.
There's no need for it as the current implementation only exists as a middle layer between the decoder and the eventual callback.

R=mflodman@webrtc.org, pbos@webrtc.org

Review URL: https://codereview.webrtc.org/2039053002 .

Cr-Commit-Position: refs/heads/master@{#13101}
2016-06-10 15:38:32 +00:00
6f8d686d35 Remove use of RtpHeaderExtension and clean up
Currently there are two structs that are identical and track extension details:
webrtc::RtpExtension
cricket::RtpHeaderExtension

The use of the structs is mixed in the code to track the extensions being
supported. This results in duplicate definition of
the URI constants and there is code to convert between the two structs.

Clean up to use a single RtpHeader throughout the codebase. The actual location
of RtpHeader may change in future (perhaps to be located in api/). Additionally,
this CL renames some of the constants to clarify Uri and Id use.

BUG= webrtc:5895

Review-Url: https://codereview.webrtc.org/1984983002
Cr-Commit-Position: refs/heads/master@{#12924}
2016-05-26 18:25:04 +00:00
dc7d0d2ef0 Move, almost, all receive side references to RTP to RtpStreamReceiver.
There are still a few places in VideoReceiveStream where the RTP module
is explicitly used, e.g. setting up a/v sync, but it's a bigger task to
change and that will be done in a follow up instead of in this CL.

BUG=webrtc:5838

Review-Url: https://codereview.webrtc.org/1947913002
Cr-Commit-Position: refs/heads/master@{#12642}
2016-05-06 12:32:30 +00:00
cfc8e3b9ef Removed all RTP dependencies from ViEChannel and renamed class.
ViEChannel is now called VideoStreamReceiver.

There will be a follow up CL removing all rtp references from VideoReceiveStream, but that made this CL to big and it will be done separately.

BUG=webrtc:5079

Review-Url: https://codereview.webrtc.org/1929313002
Cr-Commit-Position: refs/heads/master@{#12619}
2016-05-04 04:22:12 +00:00
30f118effd This cl deletes the class webrtc::VideoRendererCallback.
Replaced by VideoSinkInterface instead.

Also delete stream_id property of IncomingVideoStream.

BUG=webrtc:5426

Review-Url: https://codereview.webrtc.org/1813173002
Cr-Commit-Position: refs/heads/master@{#12602}
2016-05-03 08:09:17 +00:00
1ba8d39a9c Remove webrtc/stream.h and unutilized inheritance.
Removes inheritance and a virtual call. Also removes a root header that
would have needed to be moved into a subdirectory otherwise to prevent
circular dependencies.

BUG=webrtc:4243
R=kjellander@webrtc.org, solenberg@webrtc.org
TBR=mflodman@webrtc.org

Review-Url: https://codereview.webrtc.org/1924793002
Cr-Commit-Position: refs/heads/master@{#12586}
2016-05-02 03:18:36 +00:00
fa66659c6b Rename ViEReceiver and move ownership to VideoReceiveStream.
This CL will be followed up with another CL removing everything related
to RTP from ViEChannel to RtpStreamReceiver, i.e. remove ViEChannel::rtp_stream_receiver_.

BUG=5838

Review-Url: https://codereview.webrtc.org/1917363005
Cr-Commit-Position: refs/heads/master@{#12553}
2016-04-29 06:15:38 +00:00
0b25072c4e Use vcm::VideoReceiver on the receive side.
BUG=
R=perkj@webrtc.org

Review URL: https://codereview.webrtc.org/1905983002 .

Cr-Commit-Position: refs/heads/master@{#12473}
2016-04-22 16:23:26 +00:00
02b3d275a0 Reland of Deprecate VCMPacketizationCallback::SendData and use EncodedImageCallback instead. (patchset #1 id:1 of https://codereview.webrtc.org/1903193002/ )
Reason for revert:
A fix is being prepared downstream so this can now go in.

Original issue's description:
> Revert of Deprecate VCMPacketizationCallback::SendData and use EncodedImageCallback instead. (patchset #5 id:80001 of https://codereview.webrtc.org/1897233002/ )
>
> Reason for revert:
> API changes broke downstream.
>
> Original issue's description:
> > Deprecate VCMPacketizationCallback::SendData and use EncodedImageCallback instead.
> > EncodedImageCallback is used by all encoder implementations and seems to be what we should try to use in the transport.
> > EncodedImageCallback can of course be cleaned up in the future.
> >
> > This moves creation of RTPVideoHeader from the GenericEncoder to the PayLoadRouter.
> >
> > BUG=webrtc::5687
> >
> > Committed: https://crrev.com/f5d55aaecdc39e9cc66eb6e87614f04afe28f6eb
> > Cr-Commit-Position: refs/heads/master@{#12436}
>
> TBR=stefan@webrtc.org,pbos@webrtc.org,perkj@webrtc.org
> # Skipping CQ checks because original CL landed less than 1 days ago.
> NOPRESUBMIT=true
> NOTREECHECKS=true
> NOTRY=true
> BUG=webrtc:5687
>
> Committed: https://crrev.com/a261e6136655af33f283eda8e60a6dd93dd746a4
> Cr-Commit-Position: refs/heads/master@{#12441}

TBR=stefan@webrtc.org,pbos@webrtc.org,perkj@webrtc.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG=webrtc:5687

Review URL: https://codereview.webrtc.org/1905583002

Cr-Commit-Position: refs/heads/master@{#12442}
2016-04-20 12:06:01 +00:00
a261e61366 Revert of Deprecate VCMPacketizationCallback::SendData and use EncodedImageCallback instead. (patchset #5 id:80001 of https://codereview.webrtc.org/1897233002/ )
Reason for revert:
API changes broke downstream.

Original issue's description:
> Deprecate VCMPacketizationCallback::SendData and use EncodedImageCallback instead.
> EncodedImageCallback is used by all encoder implementations and seems to be what we should try to use in the transport.
> EncodedImageCallback can of course be cleaned up in the future.
>
> This moves creation of RTPVideoHeader from the GenericEncoder to the PayLoadRouter.
>
> BUG=webrtc::5687
>
> Committed: https://crrev.com/f5d55aaecdc39e9cc66eb6e87614f04afe28f6eb
> Cr-Commit-Position: refs/heads/master@{#12436}

TBR=stefan@webrtc.org,pbos@webrtc.org,perkj@webrtc.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG=webrtc::5687

Review URL: https://codereview.webrtc.org/1903193002

Cr-Commit-Position: refs/heads/master@{#12441}
2016-04-20 11:13:30 +00:00
f5d55aaecd Deprecate VCMPacketizationCallback::SendData and use EncodedImageCallback instead.
EncodedImageCallback is used by all encoder implementations and seems to be what we should try to use in the transport.
EncodedImageCallback can of course be cleaned up in the future.

This moves creation of RTPVideoHeader from the GenericEncoder to the PayLoadRouter.

BUG=webrtc::5687

Review URL: https://codereview.webrtc.org/1897233002

Cr-Commit-Position: refs/heads/master@{#12436}
2016-04-20 08:17:11 +00:00
a186288fd0 Revert of Update histogram "WebRTC.Video.OnewayDelayInMs" to use the estimated one-way delay. (patchset #4 id:60001 of https://codereview.webrtc.org/1688143003/ )
Reason for revert:
The delay stats are high.

Original issue's description:
> Update histogram "WebRTC.Video.OnewayDelayInMs" to use the estimated one-way delay.
> Previous logged delay was: network delay (rtt/2) + jitter delay + decode time + render delay.
>
> Make capture time in local timebase available for decoded VP9 video frames (propagate ntp_time_ms from EncodedImage to decoded VideoFrame).
>
> BUG=
>
> Committed: https://crrev.com/5249599a9b69ad9c2d513210d694719f1011f977
> Cr-Commit-Position: refs/heads/master@{#11901}

TBR=stefan@webrtc.org,pbos@webrtc.org
# Not skipping CQ checks because original CL landed more than 1 days ago.
BUG=chromium:603838

Review URL: https://codereview.webrtc.org/1893543003

Cr-Commit-Position: refs/heads/master@{#12400}
2016-04-18 07:41:09 +00:00
Per
83d0910694 Move Ownership of RtpModules to VideoSendStream from VieChannel and remove use of vie_channel and vie_receiver from video_send_stream.
The purpose of this refactoring is a first step of separating the encoder parts from the RTP transport.

BUG=webrtc:5687
R=mflodman@webrtc.org, pbos@webrtc.org, stefan@webrtc.org

Review URL: https://codereview.webrtc.org/1864313003 .

Cr-Commit-Position: refs/heads/master@{#12377}
2016-04-15 12:59:21 +00:00
26acec4772 Delete method webrtc::VideoFrame::native_handle.
Instead, use the corresponding method on VideoFrameBuffer. In the process,
reduce code duplication in frame comparison functions used in
the test code.

Make FramesEqual use FrameBufsEqual. Make the latter support texture frames.

The cl also refactors VideoFrame::CopyFrame to use I420Buffer::Copy. This
has possibly undesired side effects of never reusing the frame buffer of
the destination frame, and producing a frame buffer which may use different
stride than the source frame.

BUG=webrtc:5682

Review URL: https://codereview.webrtc.org/1881953002

Cr-Commit-Position: refs/heads/master@{#12373}
2016-04-15 10:43:45 +00:00
3911c26bc0 Add support for writing raw encoder output to .ivf files.
Also refactor GenericEncoder to use these file writers, and remove use
of preprocessor to enable file writing.

BUG=

Review URL: https://codereview.webrtc.org/1853813002

Cr-Commit-Position: refs/heads/master@{#12372}
2016-04-15 08:24:21 +00:00
74f6e9ea23 Replace NULL with nullptr in webrtc/video.
BUG=
R=danilchap@webrtc.org

Review URL: https://codereview.webrtc.org/1855433002 .

Cr-Commit-Position: refs/heads/master@{#12218}
2016-04-04 15:56:22 +00:00
7ade7b3282 Delete class webrtc::VideoRenderer and its header file.
To replace the SmoothsRenderedFrames method, added a corresponding
flag to VideoReceiveStream::Config instead.

BUG=webrtc:5426

Review URL: https://codereview.webrtc.org/1818023002

Cr-Commit-Position: refs/heads/master@{#12102}
2016-03-23 11:48:17 +00:00
eb83a1a10f This is an initial cleanup step, aiming to delete the
webrtc::VideoRenderer class, replacing it by rtc::VideoSinkInterface.

The next step is to convert all places where a renderer is attached to
rtc::VideoSourceInterface, and at that point, the
SmoothsRenderedFrames method can be replaced by a flag
rtc::VideoSinkWants::smoothed_frames.

Delete unused method IsTextureSupported.
Delete unused time argument to RenderFrame.
Let webrtc::VideoRenderer inherit rtc::VideoSinkInterface. Rename RenderFrame --> OnFrame.

TBR=kjellander@webrtc.org
BUG=webrtc:5426

Review URL: https://codereview.webrtc.org/1814763002

Cr-Commit-Position: refs/heads/master@{#12070}
2016-03-21 08:28:06 +00:00