In FrameBuffer3Proxy, if the stream became undecodable for a long
period of time and during this period the FPS changed,
the render times and decode delays would stray and cause
video pauses. This was because FrameBuffer3Proxy only updated the rtp
timestamp extrapolator on each new decodable temporal unit, rather than
each new frame.
Bug: webrtc:14168
Change-Id: I67a2c9ea392d24f84e82aa04f8c3076de11732af
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/265388
Commit-Queue: Evan Shrubsole <eshr@webrtc.org>
Reviewed-by: Philip Eliasson <philipel@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#37201}
When a large queue of frames builds up due to a lost frame, the decode
delay can sometimes become quite large. In this case the stream may
signal as timed out when in fact it is not. Instead, the delay should
be capped at the timeout limit.
Bug: webrtc:14168
Change-Id: I5b4e8851b2c6d7d27a698627dc1633931d7fc00e
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/265404
Reviewed-by: Erik Språng <sprang@webrtc.org>
Commit-Queue: Evan Shrubsole <eshr@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#37199}
FrameBuffer3Proxy and sync decoding has been shown to work. First step of cleaning up is to remove the FrameBuffer2Proxy.
Change-Id: Ic96303c2d4f9111cfeed9927e8826ea7ffe7ee17
Bug: webrtc:14003
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/264126
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Commit-Queue: Evan Shrubsole <eshr@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#37086}
Putting these classes in a sub folder increases
structure and clarifies that they are used as
helper classes. Affected classes in this change:
* CodecTimer
* InterFrameDelay
* RttFilter
VCMTiming will be moved in a separate CL.
Additional changes:
* Remove VCM prefix from class names.
* Introduce granular BUILD.gn targets.
* Update some includes.
Bug: webrtc:14111
Change-Id: Ia75128aa955a819033b97d4784cb61904de7230b
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/262960
Commit-Queue: Rasmus Brandt <brandtr@webrtc.org>
Reviewed-by: Tomas Gunnarsson <tommi@webrtc.org>
Reviewed-by: Åsa Persson <asapersson@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#36975}
The webrtc::VideoStreamDecoderInterface was basically created as a public version of FrameBuffer2, but to hide the complexity of FrameBuffer2 it was also combined with decoding so that the public API could be reasonably simple to use. FrameBuffer3 has a simple API with a clear purpose, so its API can be exposed directly.
Bug: webrtc:14026
Change-Id: I81dc84b869e4d16c5e02feb5c876fbcede3d4a25
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/261181
Reviewed-by: Tomas Gunnarsson <tommi@webrtc.org>
Commit-Queue: Philip Eliasson <philipel@webrtc.org>
Reviewed-by: Harald Alvestrand <hta@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#36781}
The field trial will remain as a kill-switch for a few weeks while
decisions about sync decoding are being made.
Change-Id: I6034d25a129404e94ab8830f51e83667e285c785
Bug: webrtc:14003
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/260327
Reviewed-by: Philip Eliasson <philipel@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Commit-Queue: Evan Shrubsole <eshr@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#36760}
This patch takes a stab at modules/video_coding,
but reaches only about half.
Bug: webrtc:10335
Change-Id: I0d47d0468b818145470c51ae4e8e75ff58d499ae
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/256112
Reviewed-by: Harald Alvestrand <hta@webrtc.org>
Commit-Queue: Jonas Oreland <jonaso@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#36335}
Additionally,
* Moved to its own GN target.
* Added unittests.
* Removed unused variable `_zeroWallClock`.
* Renamed variables to match style guide.
* Moved fields _dTS and _wrapArounds to variables.
Change-Id: I7aa8b8dec55abab49ceabe838dabf2a7e13d685d
Bug: webrtc:13756
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/253580
Reviewed-by: Niels Moller <nisse@webrtc.org>
Reviewed-by: Erik Språng <sprang@webrtc.org>
Reviewed-by: Harald Alvestrand <hta@webrtc.org>
Commit-Queue: Evan Shrubsole <eshr@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#36147}
* Uses DataSize to represent incoming and outgoing bytes.
* Puts units into doubles as they enter the Kalman filter
* Moved to its own GN target.
Change-Id: I1e7d5486a00a7158d418f553a6c77f9dd56bf3c2
Bug: webrtc:13756
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/253121
Reviewed-by: Åsa Persson <asapersson@webrtc.org>
Commit-Queue: Evan Shrubsole <eshr@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#36143}
Ensures that frames are decoded instantly when in low-latency render
mode. This also tests the max queue size behaviour. Adds a new test
suite for FrameBufferProxy that sets the appropriate field trials.
* Fixes FrameDecodeTiming to never use negative wait times for decode
timestamps.
R=kron@webrtc.org
Change-Id: I06cbec52e1e866e21aa964b24c4fd0163c26961b
Bug: webrtc:13658
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/251601
Reviewed-by: Erik Språng <sprang@webrtc.org>
Reviewed-by: Johannes Kron <kron@webrtc.org>
Commit-Queue: Evan Shrubsole <eshr@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#35999}
Adds new class DecodeSynchronizer that will coalesce the decoding
of received streams on the metronome. This feature is experimental and
is backed by a field trial WebRTC-FrameBuffer3.
This experiment now has 3 arms to it,
"WebRTC-FrameBuffer3/arm:FrameBuffer2/": Default, uses old frame buffer.
"WebRTC-FrameBuffer3/arm:FrameBuffer3/": Uses new frame buffer.
"WebRTC-FrameBuffer3/arm:SyncDecoding/": Uses new frame buffer with
frame scheduled on the metronome.
The SyncDecoding arm will not work until it is wired up in the follow-up
CL.
This change also makes the following modifications,
* Adds FakeMetronome utilities for tests using a metronome.
* Makes FrameDecodeScheduler an interface. The default implementation is
TaskQueueFrameDecodeScheduler.
* FrameDecodeScheduler now has a Stop() method, which must be called
before destruction.
TBR=philipel@webrtc.org
Change-Id: I58a306bb883604b0be3eb2a04b3d07dbdf185c71
Bug: webrtc:13658
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/250665
Reviewed-by: Henrik Boström <hbos@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Stefan Holmer <holmer@google.com>
Reviewed-by: Stefan Holmer <stefan@webrtc.org>
Commit-Queue: Evan Shrubsole <eshr@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#35988}
This emulates behaviour from frame buffer 2, but does not handle stats.
In contrast to frame buffer 2, all work happens on the same task queue.
FrameBuffer3Proxy encapsulates FrameBuffer3 and scheduler behind
a field trial WebRTC-FrameBuffer3.
This separates frame scheduling behaviour into a few components,
VideoReceiveStreamTimeoutTracker
* Handles the stream timeouts.
FrameDecodeScheduler
* Manages the scheduling and cancelling of frames being sent to the
decoder.
FrameDecodeTiming
* Handles the timing and ordering of frames to be decoded.
Other changes
* Adds CurrentSize() method to FrameBuffer3
* Move timing to a separate library
* Does a thread check for Receive statistics as this is now
on the worker thread.
* Adds `FlushImmediate` method to RunLoop so that
video_receive_stream2_unittest can pass when scheduling is happening
on the worker thread.
Change-Id: Ia8d2e5650d1708cdc1be3631a5214134583a0721
Bug: webrtc:13343
Tested: Ran webrtc_perf_tests, video_engine_tests, rtc_unittests forcing frame buffer3
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/241603
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Markus Handell <handellm@webrtc.org>
Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org>
Reviewed-by: Philip Eliasson <philipel@webrtc.org>
Reviewed-by: Tomas Gunnarsson <tommi@webrtc.org>
Commit-Queue: Evan Shrubsole <eshr@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#35847}
This is a delegate that is used by video_receive_stream2 to handle frame
buffer tasks like threading, and stats. This will be used in a follow up
to use FrameBuffer3 as a strategy selected by field trial.
Unit-tests will be used in follow-up CLs containing Frame Buffer 3, and
are expected to work with both Frame buffer proxy versions.
Change-Id: I524279343d60a348d044d9085d618f12d7bf3a23
Bug: webrtc:13343
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/241605
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Reviewed-by: Philip Eliasson <philipel@webrtc.org>
Commit-Queue: Evan Shrubsole <eshr@webrtc.org>
Cr-Commit-Position: refs/heads/main@{#35803}
