Some frames are already marked as 'timing frames' via video-timing RTP header extension. Timestamps along full WebRTC pipeline are gathered for these frames. This CL implements reporting of these timestamps for a single
timing frame since the last GetStats(). The frame with the longest end-to-end delay between two consecutive GetStats calls is reported.
The purpose of this timing information is not to provide a realtime statistics but to provide debugging information as it will help identify problematic places in video pipeline for outliers (frames which took longest to process).
BUG=webrtc:7594
Review-Url: https://codereview.webrtc.org/2946413002
Cr-Commit-Position: refs/heads/master@{#18909}
I used a command like this to update the paths:
perl -pi -e "s/webrtc\/base/webrtc\/rtc_base/g" `find webrtc/rtc_base -name "*.cc" -o -name "*.h"`
BUG=webrtc:7634
NOPRESUBMIT=True # cpplint errors that aren't caused by this CL.
Review-Url: https://codereview.webrtc.org/2969623003
Cr-Commit-Position: refs/heads/master@{#18870}
And implementation class RtpStreamReceiverController.
It's responsible for demuxing, and acts as factory for
RtpStreamReceiverInterface.
BUG=webrtc:7135
Review-Url: https://codereview.webrtc.org/2886993005
Cr-Commit-Position: refs/heads/master@{#18696}
* Change decoder thread to use new thread function type.
* Reduce the time of when video_receiver_ receives callbacks on the process thread to match with Start/Stop of the decoder.
* Not triggering shutdown unless the thread is running.
BUG=webrtc:7361
Review-Url: https://codereview.webrtc.org/2944033003
Cr-Commit-Position: refs/heads/master@{#18675}
This class is video-specific, and we want to free the name
"RtpStreamReceiver" so it can be reused for a media-independent RTP
receive class.
Also renames related files.
BUG=webrtc:7135
Review-Url: https://codereview.webrtc.org/2926253002
Cr-Commit-Position: refs/heads/master@{#18510}
To make the distinction for stats, add a |recovered| flag to
RtpPacketReceived.
BUG=webrtc:7135
Review-Url: https://codereview.webrtc.org/2693123002
Cr-Commit-Position: refs/heads/master@{#18103}
Reason for revert:
Breaks Chrome FYI Android bots.
See:
https://build.chromium.org/p/chromium.webrtc.fyi/builders/Android%20Tests%20%28dbg%29%20%28L%20Nexus9%29/builds/20418https://build.chromium.org/p/chromium.webrtc.fyi/builders/Android%20Tests%20%28dbg%29%20%28L%20Nexus6%29/builds/14724https://build.chromium.org/p/chromium.webrtc.fyi/builders/Android%20Tests%20%28dbg%29%20%28L%20Nexus5%29/builds/20133https://build.chromium.org/p/chromium.webrtc.fyi/builders/Android%20Tests%20%28dbg%29%20%28K%20Nexus5%29/builds/15111
Original issue's description:
> Deliver video frames on Android, on the decode thread.
>
> VideoCoding
> * Adding a method for polling for frames on Android only until the capture implementation takes care of this (longer term plan).
>
> CodecDatabase
> * Add an accessor for the current decoder
> * Use std::unique_ptr<> for ownership.
> * Remove "Release()" and "ReleaseDecoder()". Instead just delete.
> * Remove |friend| relationship between CodecDatabase and VCMGenericDecoder.
>
> VCMDecodedFrameCallback
> * DCHECKs for thread correctness.
> * Remove |lock_| now that a threading model has been established and verified.
>
> VCMGenericDecoder
> * All methods now have thread checks.
> * Variable access associated with thread checkers.
>
> VideoReceiver
> * Added two notification methods, DecoderThreadStarting() and DecoderThreadStopped()
> * Allows us to establish a period when the decoder thread is not running and it is safe to modify variables such as callbacks, that are only read when the decoder thread is running.
> * Allows us to DCHECK thread guarantees.
> * Allows synchronizing callbacks from the module process thread and have them only active while the decoder thread is running.
> * The above, allows us to establish two modes for the thread, single-threaded-mutable and multi-threaded-const.
> * Using that knowledge, we can remove |receive_crit_| as well as locking for a number of member variables.
>
> MediaCodecVideoDecoder
> * Removed frame polling code from this class, since this is now done from the root thread function in VideoReceiveStream.
>
> VideoReceiveStream
> * On Android: Polls for decoded frames every 10ms (same interval as previously in MediaCodecVideoDecoder)
> * [Un]Registers the |video_receiver_| with the module thread only around the time the decoder thread is started/stopped.
> * Notifies the receiver of start/stop events of the decoder thread.
> * Changed the decoder thread to use the new PlatformThread callback type.
>
> BUG=webrtc:7361, 695438
>
> Review-Url: https://codereview.webrtc.org/2764573002
> Cr-Commit-Position: refs/heads/master@{#17527}
> Committed: e3aa88bbd5TBR=sakal@webrtc.org,mflodman@webrtc.org,stefan@webrtc.org,tommi@webrtc.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG=webrtc:7361, 695438
Review-Url: https://codereview.webrtc.org/2792033003
Cr-Commit-Position: refs/heads/master@{#17530}
VideoCoding
* Adding a method for polling for frames on Android only until the capture implementation takes care of this (longer term plan).
CodecDatabase
* Add an accessor for the current decoder
* Use std::unique_ptr<> for ownership.
* Remove "Release()" and "ReleaseDecoder()". Instead just delete.
* Remove |friend| relationship between CodecDatabase and VCMGenericDecoder.
VCMDecodedFrameCallback
* DCHECKs for thread correctness.
* Remove |lock_| now that a threading model has been established and verified.
VCMGenericDecoder
* All methods now have thread checks.
* Variable access associated with thread checkers.
VideoReceiver
* Added two notification methods, DecoderThreadStarting() and DecoderThreadStopped()
* Allows us to establish a period when the decoder thread is not running and it is safe to modify variables such as callbacks, that are only read when the decoder thread is running.
* Allows us to DCHECK thread guarantees.
* Allows synchronizing callbacks from the module process thread and have them only active while the decoder thread is running.
* The above, allows us to establish two modes for the thread, single-threaded-mutable and multi-threaded-const.
* Using that knowledge, we can remove |receive_crit_| as well as locking for a number of member variables.
MediaCodecVideoDecoder
* Removed frame polling code from this class, since this is now done from the root thread function in VideoReceiveStream.
VideoReceiveStream
* On Android: Polls for decoded frames every 10ms (same interval as previously in MediaCodecVideoDecoder)
* [Un]Registers the |video_receiver_| with the module thread only around the time the decoder thread is started/stopped.
* Notifies the receiver of start/stop events of the decoder thread.
* Changed the decoder thread to use the new PlatformThread callback type.
BUG=webrtc:7361, 695438
Review-Url: https://codereview.webrtc.org/2764573002
Cr-Commit-Position: refs/heads/master@{#17527}
Use of FlexFEC is known when streams are created in
WebRtcVideoChannel2, so this replaces the code in Call to infer
FlexFEC config of video streams from the configuration of the FlexFEC
stream(s). This also allows us to switch to a more logical creation
order, where media streams are created before the FlexFEC stream.
This is done in preparation for a larger refactoring of the RTP
demuxing done in Call.
BUG=None
Review-Url: https://codereview.webrtc.org/2712683002
Cr-Commit-Position: refs/heads/master@{#17143}
This avoids redoing RTP header parsing already done in Call, for video.
The next step is to convert other types of receive streams, i.e.,
audio and flexfec, to use a compatible OnRtpPacket method. We can then
introduce a shared base interface, and simplify media-independent
receive processing in Call.
BUG=webrtc:7135
Review-Url: https://codereview.webrtc.org/2681673004
Cr-Commit-Position: refs/heads/master@{#16583}
We can then drop the CongestionController and RemoteBitrateEstimator
completely from the receive streams.
BUG=webrtc:6847
Review-Url: https://codereview.webrtc.org/2669463006
Cr-Commit-Position: refs/heads/master@{#16459}
This CL introduces a way for the VideoReceiveStreams to check whether
they are protected by any FlexfecReceiveStreams. This is done in the
VideoReceiveStream::Start() method, which then propagates this information
down to the jitter buffer adaptation logic.
BUG=webrtc:5654
Review-Url: https://codereview.webrtc.org/2649973005
Cr-Commit-Position: refs/heads/master@{#16328}
Reason for revert:
Bugfixes related to the new jitter buffer has landed.
Original issue's description:
> Revert of Make the new jitter buffer the default jitter buffer. (patchset #2 id:230001 of https://codereview.webrtc.org/2642753002/ )
>
> Reason for revert:
> Breaks tests downstream.
>
> Original issue's description:
> > Reland of Make the new jitter buffer the default jitter buffer. (patchset #1 id:1 of https://codereview.chromium.org/2632123005/ )
> >
> > Reason for revert:
> > Fix in this CL: https://codereview.chromium.org/2640793003/
> >
> > Original issue's description:
> > > Revert of Make the new jitter buffer the default jitter buffer. (patchset #7 id:120001 of https://codereview.chromium.org/2627463004/ )
> > >
> > > Reason for revert:
> > > Breaks android bots.
> > >
> > > Original issue's description:
> > > > Make the new jitter buffer the default jitter buffer.
> > > >
> > > > This CL contains only the changes necessary to make the switch to the new jitter
> > > > buffer, clean up will be done in follow up CLs.
> > > >
> > > > In this CL:
> > > > - Removed the WebRTC-NewVideoJitterBuffer experiment and made the
> > > > new video jitter buffer the default one.
> > > > - Moved WebRTC.Video.KeyFramesReceivedInPermille and
> > > > WebRTC.Video.JitterBufferDelayInMs to the ReceiveStatisticsProxy.
> > > >
> > > > BUG=webrtc:5514
> > > >
> > > > Review-Url: https://codereview.webrtc.org/2627463004
> > > > Cr-Commit-Position: refs/heads/master@{#16114}
> > > > Committed: 0f0763d86d
> > >
> > > TBR=stefan@webrtc.org,terelius@webrtc.org
> > > # Skipping CQ checks because original CL landed less than 1 days ago.
> > > NOPRESUBMIT=true
> > > NOTREECHECKS=true
> > > NOTRY=true
> > > BUG=webrtc:5514
> > >
> > > Review-Url: https://codereview.webrtc.org/2632123005
> > > Cr-Commit-Position: refs/heads/master@{#16117}
> > > Committed: c08c191f7d
> >
> > TBR=stefan@webrtc.org,terelius@webrtc.org
> > # Not skipping CQ checks because original CL landed more than 1 days ago.
> > BUG=webrtc:5514
> >
> > Review-Url: https://codereview.webrtc.org/2642753002
> > Cr-Commit-Position: refs/heads/master@{#16149}
> > Committed: f20dd0014d
>
> TBR=stefan@webrtc.org,terelius@webrtc.org,philipel@webrtc.org
> # Skipping CQ checks because original CL landed less than 1 days ago.
> NOPRESUBMIT=true
> NOTREECHECKS=true
> NOTRY=true
> BUG=webrtc:5514
>
> Review-Url: https://codereview.webrtc.org/2638423003
> Cr-Commit-Position: refs/heads/master@{#16159}
> Committed: 04926b8264TBR=stefan@webrtc.org,terelius@webrtc.org,kjellander@webrtc.org,kjellander@google.com
# Not skipping CQ checks because original CL landed more than 1 days ago.
BUG=webrtc:5514
Review-Url: https://codereview.webrtc.org/2652043005
Cr-Commit-Position: refs/heads/master@{#16293}
Order as given by inheritance in class definition.
No functional changes are intended with this CL.
BUG=None
Review-Url: https://codereview.webrtc.org/2646343005
Cr-Commit-Position: refs/heads/master@{#16272}
Reason for revert:
Breaks tests downstream.
Original issue's description:
> Reland of Make the new jitter buffer the default jitter buffer. (patchset #1 id:1 of https://codereview.chromium.org/2632123005/ )
>
> Reason for revert:
> Fix in this CL: https://codereview.chromium.org/2640793003/
>
> Original issue's description:
> > Revert of Make the new jitter buffer the default jitter buffer. (patchset #7 id:120001 of https://codereview.chromium.org/2627463004/ )
> >
> > Reason for revert:
> > Breaks android bots.
> >
> > Original issue's description:
> > > Make the new jitter buffer the default jitter buffer.
> > >
> > > This CL contains only the changes necessary to make the switch to the new jitter
> > > buffer, clean up will be done in follow up CLs.
> > >
> > > In this CL:
> > > - Removed the WebRTC-NewVideoJitterBuffer experiment and made the
> > > new video jitter buffer the default one.
> > > - Moved WebRTC.Video.KeyFramesReceivedInPermille and
> > > WebRTC.Video.JitterBufferDelayInMs to the ReceiveStatisticsProxy.
> > >
> > > BUG=webrtc:5514
> > >
> > > Review-Url: https://codereview.webrtc.org/2627463004
> > > Cr-Commit-Position: refs/heads/master@{#16114}
> > > Committed: 0f0763d86d
> >
> > TBR=stefan@webrtc.org,terelius@webrtc.org
> > # Skipping CQ checks because original CL landed less than 1 days ago.
> > NOPRESUBMIT=true
> > NOTREECHECKS=true
> > NOTRY=true
> > BUG=webrtc:5514
> >
> > Review-Url: https://codereview.webrtc.org/2632123005
> > Cr-Commit-Position: refs/heads/master@{#16117}
> > Committed: c08c191f7d
>
> TBR=stefan@webrtc.org,terelius@webrtc.org
> # Not skipping CQ checks because original CL landed more than 1 days ago.
> BUG=webrtc:5514
>
> Review-Url: https://codereview.webrtc.org/2642753002
> Cr-Commit-Position: refs/heads/master@{#16149}
> Committed: f20dd0014dTBR=stefan@webrtc.org,terelius@webrtc.org,philipel@webrtc.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG=webrtc:5514
Review-Url: https://codereview.webrtc.org/2638423003
Cr-Commit-Position: refs/heads/master@{#16159}
Reason for revert:
Fix in this CL: https://codereview.chromium.org/2640793003/
Original issue's description:
> Revert of Make the new jitter buffer the default jitter buffer. (patchset #7 id:120001 of https://codereview.chromium.org/2627463004/ )
>
> Reason for revert:
> Breaks android bots.
>
> Original issue's description:
> > Make the new jitter buffer the default jitter buffer.
> >
> > This CL contains only the changes necessary to make the switch to the new jitter
> > buffer, clean up will be done in follow up CLs.
> >
> > In this CL:
> > - Removed the WebRTC-NewVideoJitterBuffer experiment and made the
> > new video jitter buffer the default one.
> > - Moved WebRTC.Video.KeyFramesReceivedInPermille and
> > WebRTC.Video.JitterBufferDelayInMs to the ReceiveStatisticsProxy.
> >
> > BUG=webrtc:5514
> >
> > Review-Url: https://codereview.webrtc.org/2627463004
> > Cr-Commit-Position: refs/heads/master@{#16114}
> > Committed: 0f0763d86d
>
> TBR=stefan@webrtc.org,terelius@webrtc.org
> # Skipping CQ checks because original CL landed less than 1 days ago.
> NOPRESUBMIT=true
> NOTREECHECKS=true
> NOTRY=true
> BUG=webrtc:5514
>
> Review-Url: https://codereview.webrtc.org/2632123005
> Cr-Commit-Position: refs/heads/master@{#16117}
> Committed: c08c191f7dTBR=stefan@webrtc.org,terelius@webrtc.org
# Not skipping CQ checks because original CL landed more than 1 days ago.
BUG=webrtc:5514
Review-Url: https://codereview.webrtc.org/2642753002
Cr-Commit-Position: refs/heads/master@{#16149}
Reason for revert:
Breaks android bots.
Original issue's description:
> Make the new jitter buffer the default jitter buffer.
>
> This CL contains only the changes necessary to make the switch to the new jitter
> buffer, clean up will be done in follow up CLs.
>
> In this CL:
> - Removed the WebRTC-NewVideoJitterBuffer experiment and made the
> new video jitter buffer the default one.
> - Moved WebRTC.Video.KeyFramesReceivedInPermille and
> WebRTC.Video.JitterBufferDelayInMs to the ReceiveStatisticsProxy.
>
> BUG=webrtc:5514
>
> Review-Url: https://codereview.webrtc.org/2627463004
> Cr-Commit-Position: refs/heads/master@{#16114}
> Committed: 0f0763d86dTBR=stefan@webrtc.org,terelius@webrtc.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG=webrtc:5514
Review-Url: https://codereview.webrtc.org/2632123005
Cr-Commit-Position: refs/heads/master@{#16117}
This CL contains only the changes necessary to make the switch to the new jitter
buffer, clean up will be done in follow up CLs.
In this CL:
- Removed the WebRTC-NewVideoJitterBuffer experiment and made the
new video jitter buffer the default one.
- Moved WebRTC.Video.KeyFramesReceivedInPermille and
WebRTC.Video.JitterBufferDelayInMs to the ReceiveStatisticsProxy.
BUG=webrtc:5514
Review-Url: https://codereview.webrtc.org/2627463004
Cr-Commit-Position: refs/heads/master@{#16114}
In order for the VCMTiming object to be correctly updated with decoding timings
when running the WebRTC-NewVideoJitterBuffer experiment the VCMTiming object
has to be available in both the VideoReceiver and the video_coding::FrameBuffer
class. Therefore the VCMTiming object is created in VideoRecieveStream and
then passed to VideoReceiver/video_coding::FrameBuffer as they are constructed.
BUG=webrtc:5514
Review-Url: https://codereview.webrtc.org/2575473004
Cr-Commit-Position: refs/heads/master@{#15638}
And delete the method CongestionController::packet_router.
BUG=None
Review-Url: https://codereview.webrtc.org/2516983004
Cr-Commit-Position: refs/heads/master@{#15323}
Issue: video_receive_stream.cc includes transport_adapter.h which use to be inside call/ and call depends on video/ which caused circular dependency. We moved transport_adapter.h/.cc inside video/ and removed dependency of video/ on call/
BUG=webrtc:6412
NOTRY=True
Review-Url: https://codereview.webrtc.org/2470913004
Cr-Commit-Position: refs/heads/master@{#14907}
When the FlexfecReceiver recovers media packets, it inserts these into
internal::Call, which then distributes them to the appropriate
VideoReceiveStream/RtpStreamReceiver.
BUG=webrtc:5654
Review-Url: https://codereview.webrtc.org/2390823009
Cr-Commit-Position: refs/heads/master@{#14642}
After calling Start(), doing a Stop() then Start() sequence should bring
the stream back to the original state.
BUG=webrtc:6501
Review-Url: https://codereview.webrtc.org/2407163002
Cr-Commit-Position: refs/heads/master@{#14597}
This CL removes (almost) the last RTP references in VideoReceiveStream.
There are still references to RTPFragmentationHeader and SSRCs, which
will be dealt with later.
There are also new GUARDED_BY and thred checker added to the
synchronization class.
When there are othre transports than RTP, there will instead be an
interface + inheritance for RtpStreamReceiver and
RtpStreamSynchronizattion in VideoReceiveStream. This work will be done
when we actually know how we want to make thee transport interface.
BUG=webrtc:5838
Review-Url: https://codereview.webrtc.org/2216533002
Cr-Commit-Position: refs/heads/master@{#13655}
Reason for revert:
Reverting the revert. This change is not related to the failure on the Windows FYI bots. The cause of the failure has been reverted in Chromium:
https://codereview.chromium.org/2081653004/
Original issue's description:
> Revert of Split IncomingVideoStream into two implementations, with smoothing and without. (patchset #5 id:340001 of https://codereview.webrtc.org/2078873002/ )
>
> Reason for revert:
> Breaks chromium.webrtc.fyi
>
> https://uberchromegw.corp.google.com/i/chromium.webrtc.fyi/builders/Win7%20Tester/builds/4719
> https://uberchromegw.corp.google.com/i/chromium.webrtc.fyi/builders/Win10%20Tester/builds/3120
>
> Original issue's description:
> > Reland of IncomingVideoStream refactoring.
> > This reland does not contain the non-smoothing part of the original implementation. Instead, when smoothing is turned off, frame callbacks run on the decoder thread, as they did before. This code path is used in Chrome. As far as Chrome goes, the difference now is that there won't be an instance of IncomingVideoStream in between the decoder and the callback (i.e. fewer locks). Other than that, no change for Chrome.
> >
> > Original issue's description (with non-smoothing references removed):
> >
> > Split IncomingVideoStream into two implementations, with smoothing and without.
> >
> > * Added TODOs and documentation for VideoReceiveStream::OnFrame, where we today grab 6 locks.
> >
> > * Removed the Start/Stop methods from the IncomingVideoStream implementations. Now, when an instance is created, it should be considered to be "running" and when it is deleted, it's "not running". This saves on resources and also reduces the amount of locking required and I could remove one critical section altogether.
> >
> > * Changed the VideoStreamDecoder class to not depend on IncomingVideoStream but rather use the generic rtc::VideoSinkInterface<VideoFrame> interface. This means that any implementation of that interface can be used and the decoder can be made to just use the 'renderer' from the config. Once we do that, we can decouple the IncomingVideoStream implementations from the decoder and VideoReceiveStream implementations and leave it up to the application for how to do smoothing. The app can choose to use the Incoming* classes or roll its own (which may be preferable since applications often have their own scheduling mechanisms).
> >
> > * The lifetime of the VideoStreamDecoder instance is now bound to Start/Stop in VideoReceiveStream and not all of the lifetime of VideoReceiveStream.
> >
> > * Fixed VideoStreamDecoder to unregister callbacks in the dtor that were registered in the ctor. (this was open to a use-after-free regression)
> >
> > * Delay and callback pointers are now passed via the ctors to the IncomingVideoStream classes. The thread primitives in the IncomingVideoStream classes are also constructed/destructed at the same time as the owning object, which allowed me to remove one more lock.
> >
> > * Removed code in the VideoStreamDecoder that could overwrite the VideoReceiveStream render delay with a fixed value of 10ms on construction. This wasn't a problem with the previous implementation (it would be now though) but seemed to me like the wrong place to be setting that value.
> >
> > * Made the render delay value in VideoRenderFrames, const.
> >
> > BUG=chromium:620232
> > R=mflodman@webrtc.org, nisse@webrtc.org
> >
> > Committed: https://crrev.com/884c336c345d988974c2a69cea402b0fb8b07a63
> > Cr-Commit-Position: refs/heads/master@{#13219}
>
> TBR=nisse@webrtc.org,philipel@webrtc.org,mflodman@webrtc.org,tommi@webrtc.org
> # Skipping CQ checks because original CL landed less than 1 days ago.
> NOPRESUBMIT=true
> NOTREECHECKS=true
> NOTRY=true
> BUG=chromium:620232
>
> Committed: https://crrev.com/a536bfe70de38fe877245317a7f0b00bcf69cbd0
> Cr-Commit-Position: refs/heads/master@{#13229}
TBR=nisse@webrtc.org,philipel@webrtc.org,mflodman@webrtc.org,sakal@webrtc.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG=chromium:620232
Review-Url: https://codereview.webrtc.org/2089613002
Cr-Commit-Position: refs/heads/master@{#13230}
Reason for revert:
Breaks chromium.webrtc.fyi
https://uberchromegw.corp.google.com/i/chromium.webrtc.fyi/builders/Win7%20Tester/builds/4719https://uberchromegw.corp.google.com/i/chromium.webrtc.fyi/builders/Win10%20Tester/builds/3120
Original issue's description:
> Reland of IncomingVideoStream refactoring.
> This reland does not contain the non-smoothing part of the original implementation. Instead, when smoothing is turned off, frame callbacks run on the decoder thread, as they did before. This code path is used in Chrome. As far as Chrome goes, the difference now is that there won't be an instance of IncomingVideoStream in between the decoder and the callback (i.e. fewer locks). Other than that, no change for Chrome.
>
> Original issue's description (with non-smoothing references removed):
>
> Split IncomingVideoStream into two implementations, with smoothing and without.
>
> * Added TODOs and documentation for VideoReceiveStream::OnFrame, where we today grab 6 locks.
>
> * Removed the Start/Stop methods from the IncomingVideoStream implementations. Now, when an instance is created, it should be considered to be "running" and when it is deleted, it's "not running". This saves on resources and also reduces the amount of locking required and I could remove one critical section altogether.
>
> * Changed the VideoStreamDecoder class to not depend on IncomingVideoStream but rather use the generic rtc::VideoSinkInterface<VideoFrame> interface. This means that any implementation of that interface can be used and the decoder can be made to just use the 'renderer' from the config. Once we do that, we can decouple the IncomingVideoStream implementations from the decoder and VideoReceiveStream implementations and leave it up to the application for how to do smoothing. The app can choose to use the Incoming* classes or roll its own (which may be preferable since applications often have their own scheduling mechanisms).
>
> * The lifetime of the VideoStreamDecoder instance is now bound to Start/Stop in VideoReceiveStream and not all of the lifetime of VideoReceiveStream.
>
> * Fixed VideoStreamDecoder to unregister callbacks in the dtor that were registered in the ctor. (this was open to a use-after-free regression)
>
> * Delay and callback pointers are now passed via the ctors to the IncomingVideoStream classes. The thread primitives in the IncomingVideoStream classes are also constructed/destructed at the same time as the owning object, which allowed me to remove one more lock.
>
> * Removed code in the VideoStreamDecoder that could overwrite the VideoReceiveStream render delay with a fixed value of 10ms on construction. This wasn't a problem with the previous implementation (it would be now though) but seemed to me like the wrong place to be setting that value.
>
> * Made the render delay value in VideoRenderFrames, const.
>
> BUG=chromium:620232
> R=mflodman@webrtc.org, nisse@webrtc.org
>
> Committed: https://crrev.com/884c336c345d988974c2a69cea402b0fb8b07a63
> Cr-Commit-Position: refs/heads/master@{#13219}
TBR=nisse@webrtc.org,philipel@webrtc.org,mflodman@webrtc.org,tommi@webrtc.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG=chromium:620232
Review-Url: https://codereview.webrtc.org/2084873002
Cr-Commit-Position: refs/heads/master@{#13229}
This reland does not contain the non-smoothing part of the original implementation. Instead, when smoothing is turned off, frame callbacks run on the decoder thread, as they did before. This code path is used in Chrome. As far as Chrome goes, the difference now is that there won't be an instance of IncomingVideoStream in between the decoder and the callback (i.e. fewer locks). Other than that, no change for Chrome.
Original issue's description (with non-smoothing references removed):
Split IncomingVideoStream into two implementations, with smoothing and without.
* Added TODOs and documentation for VideoReceiveStream::OnFrame, where we today grab 6 locks.
* Removed the Start/Stop methods from the IncomingVideoStream implementations. Now, when an instance is created, it should be considered to be "running" and when it is deleted, it's "not running". This saves on resources and also reduces the amount of locking required and I could remove one critical section altogether.
* Changed the VideoStreamDecoder class to not depend on IncomingVideoStream but rather use the generic rtc::VideoSinkInterface<VideoFrame> interface. This means that any implementation of that interface can be used and the decoder can be made to just use the 'renderer' from the config. Once we do that, we can decouple the IncomingVideoStream implementations from the decoder and VideoReceiveStream implementations and leave it up to the application for how to do smoothing. The app can choose to use the Incoming* classes or roll its own (which may be preferable since applications often have their own scheduling mechanisms).
* The lifetime of the VideoStreamDecoder instance is now bound to Start/Stop in VideoReceiveStream and not all of the lifetime of VideoReceiveStream.
* Fixed VideoStreamDecoder to unregister callbacks in the dtor that were registered in the ctor. (this was open to a use-after-free regression)
* Delay and callback pointers are now passed via the ctors to the IncomingVideoStream classes. The thread primitives in the IncomingVideoStream classes are also constructed/destructed at the same time as the owning object, which allowed me to remove one more lock.
* Removed code in the VideoStreamDecoder that could overwrite the VideoReceiveStream render delay with a fixed value of 10ms on construction. This wasn't a problem with the previous implementation (it would be now though) but seemed to me like the wrong place to be setting that value.
* Made the render delay value in VideoRenderFrames, const.
BUG=chromium:620232
R=mflodman@webrtc.org, nisse@webrtc.org
Review URL: https://codereview.webrtc.org/2078873002 .
Cr-Commit-Position: refs/heads/master@{#13219}
Reason for revert:
Reverting again. The perf regression does not seem to be related to dropping frames.
Original issue's description:
> Reland of Split IncomingVideoStream into two implementations, with smoothing and without.
>
> Original issue's description:
>
> Split IncomingVideoStream into two implementations, with smoothing and without.
>
> This CL fixes an issue with the non-smoothing implementation where frames were delivered on the decoder thread. No-smoothing is now done in a separate class that uses a TaskQueue. The implementation may drop frames if the renderer doesn't keep up and it doesn't block the decoder thread.
>
> Further work done:
>
> * I added TODOs and documentation for VideoReceiveStream::OnFrame, where we today grab 5 locks.
>
> * I removed the Start/Stop methods from the IncomingVideoStream implementations. Now, when an instance is created, it should be considered to be "running" and when it is deleted, it's "not running". This saves on resources and also reduces the amount of locking required and I could remove one critical section altogether.
>
> * I changed the VideoStreamDecoder class to not depend on IncomingVideoStream but rather use the generic rtc::VideoSinkInterface<VideoFrame> interface. This means that any implementation of that interface can be used and the decoder can be made to just use the 'renderer' from the config. Once we do that, we can decouple the IncomingVideoStream implementations from the decoder and VideoReceiveStream implementations and leave it up to the application for how to do smoothing. The app can choose to use the Incoming* classes or roll its own (which may be preferable since applications often have their own scheduling mechanisms).
>
> * The non-smoothing IncomingVideoStream implementation currently allows only 1 outstanding pending frame. If we exceed that, the current frame won't be delivered to the renderer and instead we deliver the next one (since when this happens, the renderer is falling behind).
>
> * The lifetime of the VideoStreamDecoder instance is now bound to Start/Stop in VideoReceiveStream and not all of the lifetime of VideoReceiveStream.
>
> * Fixed VideoStreamDecoder to unregister callbacks in the dtor that were registered in the ctor. (this was open to a use-after-free regression)
>
> * Delay and callback pointers are now passed via the ctors to the IncomingVideoStream classes. The thread primitives in the IncomingVideoStream classes are also constructed/destructed at the same time as the owning object, which allowed me to remove one more lock.
>
> * Removed code in the VideoStreamDecoder that could overwrite the VideoReceiveStream render delay with a fixed value of 10ms on construction. This wasn't a problem with the previous implementation (it would be now though) but seemed to me like the wrong place to be setting that value.
>
> * Made the render delay value in VideoRenderFrames, const.
>
> BUG=chromium:620232
> TBR=mflodman
>
> Committed: https://crrev.com/e03f8787377bbc03a4e00184bb14b7561b108cbb
> Cr-Commit-Position: refs/heads/master@{#13175}
TBR=mflodman@webrtc.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG=chromium:620232
Review-Url: https://codereview.webrtc.org/2071093002
Cr-Commit-Position: refs/heads/master@{#13176}
Original issue's description:
Split IncomingVideoStream into two implementations, with smoothing and without.
This CL fixes an issue with the non-smoothing implementation where frames were delivered on the decoder thread. No-smoothing is now done in a separate class that uses a TaskQueue. The implementation may drop frames if the renderer doesn't keep up and it doesn't block the decoder thread.
Further work done:
* I added TODOs and documentation for VideoReceiveStream::OnFrame, where we today grab 5 locks.
* I removed the Start/Stop methods from the IncomingVideoStream implementations. Now, when an instance is created, it should be considered to be "running" and when it is deleted, it's "not running". This saves on resources and also reduces the amount of locking required and I could remove one critical section altogether.
* I changed the VideoStreamDecoder class to not depend on IncomingVideoStream but rather use the generic rtc::VideoSinkInterface<VideoFrame> interface. This means that any implementation of that interface can be used and the decoder can be made to just use the 'renderer' from the config. Once we do that, we can decouple the IncomingVideoStream implementations from the decoder and VideoReceiveStream implementations and leave it up to the application for how to do smoothing. The app can choose to use the Incoming* classes or roll its own (which may be preferable since applications often have their own scheduling mechanisms).
* The non-smoothing IncomingVideoStream implementation currently allows only 1 outstanding pending frame. If we exceed that, the current frame won't be delivered to the renderer and instead we deliver the next one (since when this happens, the renderer is falling behind).
* The lifetime of the VideoStreamDecoder instance is now bound to Start/Stop in VideoReceiveStream and not all of the lifetime of VideoReceiveStream.
* Fixed VideoStreamDecoder to unregister callbacks in the dtor that were registered in the ctor. (this was open to a use-after-free regression)
* Delay and callback pointers are now passed via the ctors to the IncomingVideoStream classes. The thread primitives in the IncomingVideoStream classes are also constructed/destructed at the same time as the owning object, which allowed me to remove one more lock.
* Removed code in the VideoStreamDecoder that could overwrite the VideoReceiveStream render delay with a fixed value of 10ms on construction. This wasn't a problem with the previous implementation (it would be now though) but seemed to me like the wrong place to be setting that value.
* Made the render delay value in VideoRenderFrames, const.
BUG=chromium:620232
TBR=mflodman
Review-Url: https://codereview.webrtc.org/2071473002
Cr-Commit-Position: refs/heads/master@{#13175}
