Add support for --label flag to the frame_analyzer, that
decides what label shall be used for the perf output.
BUG=none
TEST=
Make sure to have zxing and ffmpeg in the PATH.
Create a captured video (from running vie_auto_test custom call)
webrtc/tools/compare_videos.py --ref_video=reference_video.yuv --test_video=captured_output.yuv --frame_analyzer=out/Release/frame_analyzer --label=TEST_VGA
And then inspecting the output that is prefixed with RESULT.
R=phoglund@webrtc.org
Review URL: https://webrtc-codereview.appspot.com/2190005
git-svn-id: http://webrtc.googlecode.com/svn/trunk@4714 4adac7df-926f-26a2-2b94-8c16560cd09d
This is a re-land attempt of http://review.webrtc.org/1673004/
It now includes a build/isolate.gypi in WebRTC that includes the same
file as the one that would be included when WebRTC is used in a Chromium
checkout. It is needed since it is not possible to use variables in GYP's
includes sections.
Implemented according to the instructions at
http://www.chromium.org/developers/testing/isolated-testing
Workflow has been like this:
1. create _run GYP target
2. create a stripped down .isolate file
3. export GYP_DEFINES="$GYP_DEFINES test_isolation_mode=check"
4. runhooks
5. compile
6. test if the test would run (i.e. find it's dependencies) without
actually executing it:
tools/swarm_client/isolate.py run --isolated out/Release/testname.isolated
7. If failing, run the fix_test_cases.py script like this:
tools/swarm_client/googletest/fix_test_cases.py --isolated out/Release/testname.isolated
All tests that run on the bots for WebRTC has got _run target
and .isolate file created.
"Normal tests" that run fine on any machine:
* audio_decoder_unittests
* common_audio_unittests
* common_video_unittests
* metrics_unittests
* modules_tests
* modules_unittests
* neteq_unittests
* system_wrappers_unittests
* test_support_unittests
* tools_unittests
* video_engine_core_unittests
* voice_engine_unittests
Tests that requires bare-metal and audio/video devices:
* audio_device_tests
* video_capture_tests
I also added the isolate boilerplate code for the following
tests that are not yet pure gtest binaries (which means they
cannot run isolated yet):
* video_render_tests
* vie_auto_test
* voe_auto_test
TEST=running isolate.py as described above. WebRTC trybots passing. Created a Chromium checkout with third_party/webrtc ToT and this patch applied, passing the runhooks step.
BUG=1916
R=henrike@webrtc.org, tommi@webrtc.org
Review URL: https://webrtc-codereview.appspot.com/2056004
git-svn-id: http://webrtc.googlecode.com/svn/trunk@4590 4adac7df-926f-26a2-2b94-8c16560cd09d
As this breaks the FYI bots in
http://build.chromium.org/p/chromium.webrtc.fyi/waterfall
due to different path to isolate.gypi (which cannot easily
be resolved due to limitations in GYP)
> Isolate GYP target and .isolate files for tests
>
> Implemented according to the instructions at
> http://www.chromium.org/developers/testing/isolated-testing
>
> Workflow has been like this:
> 1. create _run GYP target
> 2. create a stripped down .isolate file
> 3. export GYP_DEFINES="$GYP_DEFINES test_isolation_mode=check"
> 4. runhooks
> 5. compile
> 6. test if the test would run (i.e. find it's dependencies) without
> actually executing it:
> tools/swarm_client/isolate.py run --isolated out/Release/testname.isolated
> 7. If failing, run the fix_test_cases.py script like this:
> tools/swarm_client/fix_test_cases.py --isolated out/Release/testname.isolated
>
> All tests that run on the bots for WebRTC has got _run target
> and .isolate file created.
>
> "Normal tests" that run fine on any machine:
> * audio_decoder_unittests
> * common_audio_unittests
> * common_video_unittests
> * metrics_unittests
> * modules_integrationtests
> * modules_unittests
> * neteq_unittests
> * system_wrappers_unittests
> * test_support_unittests
> * tools_unittests
> * video_engine_core_unittests
> * voice_engine_unittests
>
> Tests that requires bare-metal and audio/video devices:
> * audio_device_integrationtests
> * video_capture_integrationtests
>
> I also added the isolate boilerplate code for the following
> tests that are not yet pure gtest binaries (which means they
> cannot run isolated yet):
> * video_render_integrationtests
> * vie_auto_test
> * voe_auto_test
>
> TEST=running isolate.py as described above.
> BUG=1916
> R=tommi@webrtc.org
>
> Review URL: https://webrtc-codereview.appspot.com/1673004TBR=kjellander@webrtc.org
Review URL: https://webrtc-codereview.appspot.com/2040004
git-svn-id: http://webrtc.googlecode.com/svn/trunk@4548 4adac7df-926f-26a2-2b94-8c16560cd09d
Implemented according to the instructions at
http://www.chromium.org/developers/testing/isolated-testing
Workflow has been like this:
1. create _run GYP target
2. create a stripped down .isolate file
3. export GYP_DEFINES="$GYP_DEFINES test_isolation_mode=check"
4. runhooks
5. compile
6. test if the test would run (i.e. find it's dependencies) without
actually executing it:
tools/swarm_client/isolate.py run --isolated out/Release/testname.isolated
7. If failing, run the fix_test_cases.py script like this:
tools/swarm_client/fix_test_cases.py --isolated out/Release/testname.isolated
All tests that run on the bots for WebRTC has got _run target
and .isolate file created.
"Normal tests" that run fine on any machine:
* audio_decoder_unittests
* common_audio_unittests
* common_video_unittests
* metrics_unittests
* modules_integrationtests
* modules_unittests
* neteq_unittests
* system_wrappers_unittests
* test_support_unittests
* tools_unittests
* video_engine_core_unittests
* voice_engine_unittests
Tests that requires bare-metal and audio/video devices:
* audio_device_integrationtests
* video_capture_integrationtests
I also added the isolate boilerplate code for the following
tests that are not yet pure gtest binaries (which means they
cannot run isolated yet):
* video_render_integrationtests
* vie_auto_test
* voe_auto_test
TEST=running isolate.py as described above.
BUG=1916
R=tommi@webrtc.org
Review URL: https://webrtc-codereview.appspot.com/1673004
git-svn-id: http://webrtc.googlecode.com/svn/trunk@4547 4adac7df-926f-26a2-2b94-8c16560cd09d
Now that these scripts are called from browser tests, we need to print everything on stdout since the tests will throw away stderr when invoking programs. I chose to assign sys.stderr to sys.stdout. Otherwise I would have missed stuff like parser.error, which print to stderr.
The error message will get improved because the old code did not catch the case when the binary was missing, which lead to a very confusing error when that was the case. This gets fixed now.
BUG=
R=kjellander@webrtc.org
Review URL: https://webrtc-codereview.appspot.com/1886004
git-svn-id: http://webrtc.googlecode.com/svn/trunk@4416 4adac7df-926f-26a2-2b94-8c16560cd09d
This test was initializing strings for reference video files in a static
context, which makes is against the style guide and also makes the paths
become invalid when the test is launched from a working directory
outside the checkout.
Moving the initialization into the test fixture solves this.
BUG=none
TEST=Local execution launched from a directory outside the checkout tree on Win, Mac and Linux + trybots (for
compilation as they don't yet run the tools_unittests).
Review URL: https://webrtc-codereview.appspot.com/1178004
git-svn-id: http://webrtc.googlecode.com/svn/trunk@3647 4adac7df-926f-26a2-2b94-8c16560cd09d
By using the C++ version of Zxing, we can avoid having Java and Ant
as a dependency when running Video quality analysis on the bots.
This makes it far more easy to setup automation on new machines.
I also moved the scripts into the webrtc/ folder so it will be synced by default when building in Chrome (eliminating the need of a separate solution).
This CL also removes the need of the FFMPEG_HOME variable and replaces
its use with a command line flag to make the tool run smoothly on
Windows.
BUG=none
TEST=locally running the script on Windows, Mac and Linux.
Review URL: https://webrtc-codereview.appspot.com/1099007
git-svn-id: http://webrtc.googlecode.com/svn/trunk@3640 4adac7df-926f-26a2-2b94-8c16560cd09d
The test fails since it's assuming out/testfile.yuv exists when running the test. Just opening the file at a later time than the SetUp function seems to break the test so that's not a viable solution. This CL uses a simple workaround that simply truncates the file before opening it, which works.
BUG=none
TEST=tools_unittests in Debug+Release on Mac, Win and Linux + memcheck, tsan, asan.
Review URL: https://webrtc-codereview.appspot.com/1067004
git-svn-id: http://webrtc.googlecode.com/svn/trunk@3401 4adac7df-926f-26a2-2b94-8c16560cd09d
This was caused by not supplying a correct pointer to where fread should read. The files are now opened in binary mode (which I have under stood can cause problems between different OS if it is not done). I also check for EOF when I compare data from fread. Previously the checking for correct amount of bytes read failed when the end of the file had been reached.
BUG=
Review URL: https://webrtc-codereview.appspot.com/937032
git-svn-id: http://webrtc.googlecode.com/svn/trunk@3212 4adac7df-926f-26a2-2b94-8c16560cd09d
