Reason for revert:
Reverting again. The perf regression does not seem to be related to dropping frames.
Original issue's description:
> Reland of Split IncomingVideoStream into two implementations, with smoothing and without.
>
> Original issue's description:
>
> Split IncomingVideoStream into two implementations, with smoothing and without.
>
> This CL fixes an issue with the non-smoothing implementation where frames were delivered on the decoder thread. No-smoothing is now done in a separate class that uses a TaskQueue. The implementation may drop frames if the renderer doesn't keep up and it doesn't block the decoder thread.
>
> Further work done:
>
> * I added TODOs and documentation for VideoReceiveStream::OnFrame, where we today grab 5 locks.
>
> * I removed the Start/Stop methods from the IncomingVideoStream implementations. Now, when an instance is created, it should be considered to be "running" and when it is deleted, it's "not running". This saves on resources and also reduces the amount of locking required and I could remove one critical section altogether.
>
> * I changed the VideoStreamDecoder class to not depend on IncomingVideoStream but rather use the generic rtc::VideoSinkInterface<VideoFrame> interface. This means that any implementation of that interface can be used and the decoder can be made to just use the 'renderer' from the config. Once we do that, we can decouple the IncomingVideoStream implementations from the decoder and VideoReceiveStream implementations and leave it up to the application for how to do smoothing. The app can choose to use the Incoming* classes or roll its own (which may be preferable since applications often have their own scheduling mechanisms).
>
> * The non-smoothing IncomingVideoStream implementation currently allows only 1 outstanding pending frame. If we exceed that, the current frame won't be delivered to the renderer and instead we deliver the next one (since when this happens, the renderer is falling behind).
>
> * The lifetime of the VideoStreamDecoder instance is now bound to Start/Stop in VideoReceiveStream and not all of the lifetime of VideoReceiveStream.
>
> * Fixed VideoStreamDecoder to unregister callbacks in the dtor that were registered in the ctor. (this was open to a use-after-free regression)
>
> * Delay and callback pointers are now passed via the ctors to the IncomingVideoStream classes. The thread primitives in the IncomingVideoStream classes are also constructed/destructed at the same time as the owning object, which allowed me to remove one more lock.
>
> * Removed code in the VideoStreamDecoder that could overwrite the VideoReceiveStream render delay with a fixed value of 10ms on construction. This wasn't a problem with the previous implementation (it would be now though) but seemed to me like the wrong place to be setting that value.
>
> * Made the render delay value in VideoRenderFrames, const.
>
> BUG=chromium:620232
> TBR=mflodman
>
> Committed: https://crrev.com/e03f8787377bbc03a4e00184bb14b7561b108cbb
> Cr-Commit-Position: refs/heads/master@{#13175}
TBR=mflodman@webrtc.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG=chromium:620232
Review-Url: https://codereview.webrtc.org/2071093002
Cr-Commit-Position: refs/heads/master@{#13176}
Original issue's description:
Split IncomingVideoStream into two implementations, with smoothing and without.
This CL fixes an issue with the non-smoothing implementation where frames were delivered on the decoder thread. No-smoothing is now done in a separate class that uses a TaskQueue. The implementation may drop frames if the renderer doesn't keep up and it doesn't block the decoder thread.
Further work done:
* I added TODOs and documentation for VideoReceiveStream::OnFrame, where we today grab 5 locks.
* I removed the Start/Stop methods from the IncomingVideoStream implementations. Now, when an instance is created, it should be considered to be "running" and when it is deleted, it's "not running". This saves on resources and also reduces the amount of locking required and I could remove one critical section altogether.
* I changed the VideoStreamDecoder class to not depend on IncomingVideoStream but rather use the generic rtc::VideoSinkInterface<VideoFrame> interface. This means that any implementation of that interface can be used and the decoder can be made to just use the 'renderer' from the config. Once we do that, we can decouple the IncomingVideoStream implementations from the decoder and VideoReceiveStream implementations and leave it up to the application for how to do smoothing. The app can choose to use the Incoming* classes or roll its own (which may be preferable since applications often have their own scheduling mechanisms).
* The non-smoothing IncomingVideoStream implementation currently allows only 1 outstanding pending frame. If we exceed that, the current frame won't be delivered to the renderer and instead we deliver the next one (since when this happens, the renderer is falling behind).
* The lifetime of the VideoStreamDecoder instance is now bound to Start/Stop in VideoReceiveStream and not all of the lifetime of VideoReceiveStream.
* Fixed VideoStreamDecoder to unregister callbacks in the dtor that were registered in the ctor. (this was open to a use-after-free regression)
* Delay and callback pointers are now passed via the ctors to the IncomingVideoStream classes. The thread primitives in the IncomingVideoStream classes are also constructed/destructed at the same time as the owning object, which allowed me to remove one more lock.
* Removed code in the VideoStreamDecoder that could overwrite the VideoReceiveStream render delay with a fixed value of 10ms on construction. This wasn't a problem with the previous implementation (it would be now though) but seemed to me like the wrong place to be setting that value.
* Made the render delay value in VideoRenderFrames, const.
BUG=chromium:620232
TBR=mflodman
Review-Url: https://codereview.webrtc.org/2071473002
Cr-Commit-Position: refs/heads/master@{#13175}
Reason for revert:
Reverting while we track down the issue on the Win10 bot.
Original issue's description:
> Split IncomingVideoStream into two implementations, with smoothing and without.
>
> This CL fixes an issue with the non-smoothing implementation where frames were delivered on the decoder thread. No-smoothing is now done in a separate class that uses a TaskQueue. The implementation may drop frames if the renderer doesn't keep up and it doesn't block the decoder thread.
>
> Further work done:
>
> * I added TODOs and documentation for VideoReceiveStream::OnFrame, where we today grab 5 locks.
>
> * I removed the Start/Stop methods from the IncomingVideoStream implementations. Now, when an instance is created, it should be considered to be "running" and when it is deleted, it's "not running". This saves on resources and also reduces the amount of locking required and I could remove one critical section altogether.
>
> * I changed the VideoStreamDecoder class to not depend on IncomingVideoStream but rather use the generic rtc::VideoSinkInterface<VideoFrame> interface. This means that any implementation of that interface can be used and the decoder can be made to just use the 'renderer' from the config. Once we do that, we can decouple the IncomingVideoStream implementations from the decoder and VideoReceiveStream implementations and leave it up to the application for how to do smoothing. The app can choose to use the Incoming* classes or roll its own (which may be preferable since applications often have their own scheduling mechanisms).
>
> * The non-smoothing IncomingVideoStream implementation currently allows only 1 outstanding pending frame. If we exceed that, the current frame won't be delivered to the renderer and instead we deliver the next one (since when this happens, the renderer is falling behind).
>
> * The lifetime of the VideoStreamDecoder instance is now bound to Start/Stop in VideoReceiveStream and not all of the lifetime of VideoReceiveStream.
>
> * Fixed VideoStreamDecoder to unregister callbacks in the dtor that were registered in the ctor. (this was open to a use-after-free regression)
>
> * Delay and callback pointers are now passed via the ctors to the IncomingVideoStream classes. The thread primitives in the IncomingVideoStream classes are also constructed/destructed at the same time as the owning object, which allowed me to remove one more lock.
>
> * Removed code in the VideoStreamDecoder that could overwrite the VideoReceiveStream render delay with a fixed value of 10ms on construction. This wasn't a problem with the previous implementation (it would be now though) but seemed to me like the wrong place to be setting that value.
>
> * Made the render delay value in VideoRenderFrames, const.
>
> BUG=
>
> Committed: https://crrev.com/1c7eef652b0aa22d8ebb0bfe2b547094a794be22
> Cr-Commit-Position: refs/heads/master@{#13129}
TBR=mflodman@webrtc.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG=
Review-Url: https://codereview.webrtc.org/2061363002
Cr-Commit-Position: refs/heads/master@{#13146}
This CL fixes an issue with the non-smoothing implementation where frames were delivered on the decoder thread. No-smoothing is now done in a separate class that uses a TaskQueue. The implementation may drop frames if the renderer doesn't keep up and it doesn't block the decoder thread.
Further work done:
* I added TODOs and documentation for VideoReceiveStream::OnFrame, where we today grab 5 locks.
* I removed the Start/Stop methods from the IncomingVideoStream implementations. Now, when an instance is created, it should be considered to be "running" and when it is deleted, it's "not running". This saves on resources and also reduces the amount of locking required and I could remove one critical section altogether.
* I changed the VideoStreamDecoder class to not depend on IncomingVideoStream but rather use the generic rtc::VideoSinkInterface<VideoFrame> interface. This means that any implementation of that interface can be used and the decoder can be made to just use the 'renderer' from the config. Once we do that, we can decouple the IncomingVideoStream implementations from the decoder and VideoReceiveStream implementations and leave it up to the application for how to do smoothing. The app can choose to use the Incoming* classes or roll its own (which may be preferable since applications often have their own scheduling mechanisms).
* The non-smoothing IncomingVideoStream implementation currently allows only 1 outstanding pending frame. If we exceed that, the current frame won't be delivered to the renderer and instead we deliver the next one (since when this happens, the renderer is falling behind).
* The lifetime of the VideoStreamDecoder instance is now bound to Start/Stop in VideoReceiveStream and not all of the lifetime of VideoReceiveStream.
* Fixed VideoStreamDecoder to unregister callbacks in the dtor that were registered in the ctor. (this was open to a use-after-free regression)
* Delay and callback pointers are now passed via the ctors to the IncomingVideoStream classes. The thread primitives in the IncomingVideoStream classes are also constructed/destructed at the same time as the owning object, which allowed me to remove one more lock.
* Removed code in the VideoStreamDecoder that could overwrite the VideoReceiveStream render delay with a fixed value of 10ms on construction. This wasn't a problem with the previous implementation (it would be now though) but seemed to me like the wrong place to be setting that value.
* Made the render delay value in VideoRenderFrames, const.
BUG=
Review-Url: https://codereview.webrtc.org/2035173002
Cr-Commit-Position: refs/heads/master@{#13129}
Introduce a new method I420Buffer::CropAndScale, and a static
convenience helper I420Buffer::CenterCropAndScale. Use them for almost
all scaling needs.
Delete the Scaler class and the cricket::VideoFrame::Stretch* methods.
BUG=webrtc:5682
R=pbos@webrtc.org, perkj@webrtc.org, stefan@webrtc.org
Review URL: https://codereview.webrtc.org/2020593002 .
Cr-Commit-Position: refs/heads/master@{#13110}
The VUI part an SPS may specify max_num_reorder_frames and
max_dec_frame_buffering. These may cause a decoder to buffer a number
of frame prior allowing decode, leading to delays, even if no frames
using such references (ie B-frames) are sent.
Because of this we update any SPS block emitted by the encoder.
Also, a bunch of refactoring of H264-related code to reduce code
duplication.
BUG=
Review-Url: https://codereview.webrtc.org/1979443004
Cr-Commit-Position: refs/heads/master@{#13010}
We plan to add junit tests running with Robolectric
so naming these files "apk" is slightly confusing.
NOTRY=True
Review-Url: https://codereview.webrtc.org/2020213002
Cr-Commit-Position: refs/heads/master@{#12971}
Reason for revert:
Breaks downstream compilation. Please make non-breaking API changes for the reland or coordinate fixing downstream code quickly with the sheriff.
Original issue's description:
> Cleanup of webrtc::VideoFrame.
>
> Delete EqualsFrame method, used only by tests. Delete one of the
> CreateFrame methods. Drop return value for CreateEmptyFrame, CreateFrame
> and CopyFrame.
>
> BUG=webrtc:5426
>
> Committed: https://crrev.com/208019637bfed975f8f13b16d40b90e200763cd6
> Cr-Commit-Position: refs/heads/master@{#11783}
TBR=pbos@webrtc.org,perkj@webrtc.org,pthatcher@webrtc.org,mflodman@webrtc.org,marpan@webrtc.org,nisse@webrtc.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG=webrtc:5426
Review URL: https://codereview.webrtc.org/1743613002
Cr-Commit-Position: refs/heads/master@{#11789}
Delete EqualsFrame method, used only by tests. Delete one of the
CreateFrame methods. Drop return value for CreateEmptyFrame, CreateFrame
and CopyFrame.
BUG=webrtc:5426
Review URL: https://codereview.webrtc.org/1679323002
Cr-Commit-Position: refs/heads/master@{#11783}
With this change the following tests have been successfully
passing in the iOS Simulator for iPhone 5 and iOS 9:
* audio_decoder_unittests
* common_video_unittests
* modules_tests
* rtc_api_objc_tests
* rtc_pc_unittests
* system_wrappers_unittests
* voice_engine_unittests
The modules_unittests and common_audio_unittests are
handled in https://codereview.webrtc.org/1698033002/
BUG=webrtc:4755
NOTRY=True
Review URL: https://codereview.webrtc.org/1694353003
Cr-Commit-Position: refs/heads/master@{#11646}
Renames CreateFakeNativeHandleFrame to FakeNativeHandle::CreateFrame and
moves into test.gyp target 'fake_video_frames' which contains previous
frame_generator target.
Removes unused warnings from includers of
webrtc/test/fake_texture_frame.h which did not use the function above.
BUG=webrtc:5398
R=kjellander@webrtc.orgTBR=stefan@webrtc.org
Review URL: https://codereview.webrtc.org/1554223002 .
Cr-Commit-Position: refs/heads/master@{#11149}
Now when we don't use SwapFrame consistently anymore, we need to recycle allocations with a buffer pool instead. This CL adds a buffer pool class, and updates the vp8 decoder to use it. If this CL lands successfully I will update the other video producers as well.
BUG=1128
R=stefan@webrtc.org, tommi@webrtc.org
Review URL: https://webrtc-codereview.appspot.com/41189004
Cr-Commit-Position: refs/heads/master@{#8748}
git-svn-id: http://webrtc.googlecode.com/svn/trunk@8748 4adac7df-926f-26a2-2b94-8c16560cd09d
TextureVideoFrame is currently an empty shell that only provides a convenience constructor of I420VideoFrame with a texture buffer. This CL moves that constructor, and all unittests, of TextureVideoFrame into the base class. Then it's possible to completely remove TextureVideoFrame and all its files. Also, there is no point in having I420VideoFrame virtual anymore.
R=pbos@webrtc.org, perkj@webrtc.org, stefan@webrtc.org
TBR=mflodman
Review URL: https://webrtc-codereview.appspot.com/40229004
Cr-Commit-Position: refs/heads/master@{#8629}
git-svn-id: http://webrtc.googlecode.com/svn/trunk@8629 4adac7df-926f-26a2-2b94-8c16560cd09d
It's possible to build Chrome on Windows with this patch now.
BUG=1128
> This is unfortunately causing build problems in Chrome on Windows.
>> Unify underlying frame buffer in I420VideoFrame and WebRtcVideoFrame
>>
>> Currently, I420VideoFrame uses three webrtc::Plane to store pixel data, and WebRtcVideoFrame uses WebRtcVideoFrame::FrameBuffer/webrtc::VideoFrame. The two subclasses WebRtcTextureVideoFrame and TextureVideoFrame use a NativeHandle to store pixel data, and there is also a class WebRtcVideoRenderFrame that wraps an I420VideoFrame.
>>
>> This CL replaces these classes with a new interface VideoFrameBuffer that provides the common functionality. This makes it possible to remove deep frame copies between cricket::VideoFrame and I420VideoFrame.
>>
>> Some additional minor changes are:
>> * Disallow creation of 0x0 texture frames.
>> * Remove the half-implemented ref count functions in I420VideoFrame.
>> * Remove the Alias functionality in WebRtcVideoFrame
>>
>> The final goal is to eliminate all frame copies, but to limit the scope of this CL, some planned changes are postponed to follow-up CL:s (see planned changes in https://webrtc-codereview.appspot.com/38879004, or https://docs.google.com/document/d/1bxoJZNmlo-Z9GnQwIaWpEG6hDlL_W-bzka8Zb_K2NbA/preview). Specifically, this CL:
>> * Keeps empty subclasses WebRtcTextureVideoFrame and TextureVideoFrame, and just delegates the construction to the superclass.
>> * Keeps the deep copies from cricket::VideoFrame to I420VideoFrame.
>>
>> BUG=1128
>> R=mflodman@webrtc.org, pbos@webrtc.org, perkj@webrtc.org, tommi@webrtc.org
>>
>> Review URL: https://webrtc-codereview.appspot.com/42469004R=pbos@webrtc.org
TBR=mflodman, pbos, perkj, tommi
Review URL: https://webrtc-codereview.appspot.com/45489004
Cr-Commit-Position: refs/heads/master@{#8616}
git-svn-id: http://webrtc.googlecode.com/svn/trunk@8616 4adac7df-926f-26a2-2b94-8c16560cd09d
This is unfortunately causing build problems in Chrome on Windows.
> Unify underlying frame buffer in I420VideoFrame and WebRtcVideoFrame
>
> Currently, I420VideoFrame uses three webrtc::Plane to store pixel data, and WebRtcVideoFrame uses WebRtcVideoFrame::FrameBuffer/webrtc::VideoFrame. The two subclasses WebRtcTextureVideoFrame and TextureVideoFrame use a NativeHandle to store pixel data, and there is also a class WebRtcVideoRenderFrame that wraps an I420VideoFrame.
>
> This CL replaces these classes with a new interface VideoFrameBuffer that provides the common functionality. This makes it possible to remove deep frame copies between cricket::VideoFrame and I420VideoFrame.
>
> Some additional minor changes are:
> * Disallow creation of 0x0 texture frames.
> * Remove the half-implemented ref count functions in I420VideoFrame.
> * Remove the Alias functionality in WebRtcVideoFrame
>
> The final goal is to eliminate all frame copies, but to limit the scope of this CL, some planned changes are postponed to follow-up CL:s (see planned changes in https://webrtc-codereview.appspot.com/38879004, or https://docs.google.com/document/d/1bxoJZNmlo-Z9GnQwIaWpEG6hDlL_W-bzka8Zb_K2NbA/preview). Specifically, this CL:
> * Keeps empty subclasses WebRtcTextureVideoFrame and TextureVideoFrame, and just delegates the construction to the superclass.
> * Keeps the deep copies from cricket::VideoFrame to I420VideoFrame.
>
> BUG=1128
> R=mflodman@webrtc.org, pbos@webrtc.org, perkj@webrtc.org, tommi@webrtc.org
>
> Review URL: https://webrtc-codereview.appspot.com/42469004TBR=magjed@webrtc.org
Review URL: https://webrtc-codereview.appspot.com/42199005
Cr-Commit-Position: refs/heads/master@{#8599}
git-svn-id: http://webrtc.googlecode.com/svn/trunk@8599 4adac7df-926f-26a2-2b94-8c16560cd09d
Currently, I420VideoFrame uses three webrtc::Plane to store pixel data, and WebRtcVideoFrame uses WebRtcVideoFrame::FrameBuffer/webrtc::VideoFrame. The two subclasses WebRtcTextureVideoFrame and TextureVideoFrame use a NativeHandle to store pixel data, and there is also a class WebRtcVideoRenderFrame that wraps an I420VideoFrame.
This CL replaces these classes with a new interface VideoFrameBuffer that provides the common functionality. This makes it possible to remove deep frame copies between cricket::VideoFrame and I420VideoFrame.
Some additional minor changes are:
* Disallow creation of 0x0 texture frames.
* Remove the half-implemented ref count functions in I420VideoFrame.
* Remove the Alias functionality in WebRtcVideoFrame
The final goal is to eliminate all frame copies, but to limit the scope of this CL, some planned changes are postponed to follow-up CL:s (see planned changes in https://webrtc-codereview.appspot.com/38879004, or https://docs.google.com/document/d/1bxoJZNmlo-Z9GnQwIaWpEG6hDlL_W-bzka8Zb_K2NbA/preview). Specifically, this CL:
* Keeps empty subclasses WebRtcTextureVideoFrame and TextureVideoFrame, and just delegates the construction to the superclass.
* Keeps the deep copies from cricket::VideoFrame to I420VideoFrame.
BUG=1128
R=mflodman@webrtc.org, pbos@webrtc.org, perkj@webrtc.org, tommi@webrtc.org
Review URL: https://webrtc-codereview.appspot.com/42469004
Cr-Commit-Position: refs/heads/master@{#8580}
git-svn-id: http://webrtc.googlecode.com/svn/trunk@8580 4adac7df-926f-26a2-2b94-8c16560cd09d
This reverts selected parts of r7014 to enable
rolling WebRTC in Chromium DEPS.
This works around the problem with GYP includes
being processed in the first pass (i.e. variables
cannot be used for paths). Using a dependency with
a path using a variable that is conditioned for
build_with_chromium being 0 or 1 solves the Chromium
build.
These changes will be restored once I've finished
a major GYP refactoring that will break out all
test related code (at least the parts that includes
the Android APK targets) into a separate chain
of GYP targets that are not processed when generating
projects for Chromium (which is why r7014 is breaking
the Chromium build).
BUG=3741
TESTED=Passing compilation of standalone using:
GYP_DEFINES="OS=android component=static_library fastbuild=1 target_arch=arm" webrtc/build/gyp_webrtc
ninja -C out/Debug
Then verified the *_apk targets are generated and compiled.
Passing compilation from a Chromium checkout with third_party/webrtc
directory removed and a new empty third_party/webrtc mapped to the
standalone checkout using:
sudo mount --bind /path/to/trunk/webrtc third_party/webrtc
Then running build/gyp_chromium
I also verified WebRTC GYP targets exist and are able to compile.
R=henrike@webrtc.org
Review URL: https://webrtc-codereview.appspot.com/20299004
git-svn-id: http://webrtc.googlecode.com/svn/trunk@7040 4adac7df-926f-26a2-2b94-8c16560cd09d
Restructure how the Android APK tests are compiled now
that we have a Chromium checkout available (since r6938).
This removes the need of several hacks that were needed when
building these targets from inside a Chromium checkout.
By creating a symlink to Chromium's base we can compile the required
targets. This also removes the need of the previously precompiled
binaries we keep in /deps/tools/android at Google code.
All the user needs to do is to add the target_os = ["android"]
entry to his .gclient as described at
https://code.google.com/p/chromium/wiki/AndroidBuildInstructions
Before committing this CL, the Android APK buildbots will need
to be updated.
This also solves http://crbug.com/402594 since the apply_svn_patch.py
usage will be similar to the other standalone bots.
It also solves http://crbug.com/399297
BUG=chromium:399297, chromium:402594
TESTED=Locally compiled all APK targets by running:
GYP_DEFINES="OS=android include_tests=1 enable_tracing=1" gclient runhooks
ninja -C out/Release
checkdeps
R=henrike@webrtc.org, tommi@webrtc.org
Review URL: https://webrtc-codereview.appspot.com/22149004
git-svn-id: http://webrtc.googlecode.com/svn/trunk@7014 4adac7df-926f-26a2-2b94-8c16560cd09d
Optionally prevents doing a frame copy when putting frames into a
VideoSendStream. PutFrame() is still there, which copies the frame.
Also removes time_since_capture_ms as a parameter, since
I420VideoFrame::render_time_ms() denotes when the frame was captured.
BUG=2657
R=mflodman@webrtc.org
Review URL: https://webrtc-codereview.appspot.com/5119004
git-svn-id: http://webrtc.googlecode.com/svn/trunk@5265 4adac7df-926f-26a2-2b94-8c16560cd09d
