It will be easier to make perf results output to JSON if the PrintResult*
functions receive doubles instead of strings.
I'll make follow-up CLs for PrintResultMeanAndError and PrintResultList.
Bug: webrtc:8566
Change-Id: I198e422a7bb8cd237c6364af98d2f67f0858452e
Reviewed-on: https://webrtc-review.googlesource.com/25300
Reviewed-by: Patrik Höglund <phoglund@webrtc.org>
Reviewed-by: Björn Terelius <terelius@webrtc.org>
Commit-Queue: Edward Lemur <ehmaldonado@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#20848}
Calculation of quality metrics required writing of decoded video
to file. There were two drawbacks with that approach. First, frame
drops significantly affected metrics because comparison was done
against the last decoded frame. Second, simulcast/SVC required
writing of multiple files. This might be too much data to dump.
On-fly metrics calculation is done in frame decoded callback.
Calculation time is excluded from encoding/decoding time. If CPU
usage measurement is enabled metrics calculation is disabled since
it affects CPU usage. The results are reported in Stats::PrintSummary.
Bug: webrtc:8524
Change-Id: Id54fb21f2f95deeb93757afaf46bde7d7ae18dac
Reviewed-on: https://webrtc-review.googlesource.com/22560
Commit-Queue: Sergey Silkin <ssilkin@webrtc.org>
Reviewed-by: Patrik Höglund <phoglund@webrtc.org>
Reviewed-by: Rasmus Brandt <brandtr@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#20798}
On android, the flag to store the frame with the worst PSNR was called
'--test_artifacts_dir'.
I think test artifacts is a better name.
TBR=sprang@webrtc.org
Bug: chromium:745469
Change-Id: I358ea2985a1df2da12b81df173d74ac193556a49
Reviewed-on: https://webrtc-review.googlesource.com/4080
Commit-Queue: Edward Lemur <ehmaldonado@webrtc.org>
Reviewed-by: Henrik Kjellander <kjellander@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#20000}
Now that we have moved WebRTC from src/webrtc to src/, common_types.h
and typedefs.h are triggering a cpplint error.
The cpplint complaint is:
Include the directory when naming .h files [build/include] [4]
This CL disables the error but we have to remove these two headers
from the root directory.
NOPRESUBMIT=true
Bug: webrtc:5876
Change-Id: I08e1b69aadcc4b28ab83bf25e3819d135d41d333
Reviewed-on: https://webrtc-review.googlesource.com/1577
Commit-Queue: Mirko Bonadei <mbonadei@webrtc.org>
Reviewed-by: Henrik Kjellander <kjellander@google.com>
Reviewed-by: Karl Wiberg <kwiberg@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#19859}
In order to eliminate the WebRTC Subtree mirror in Chromium,
WebRTC is moving the content of the src/webrtc directory up
to the src/ directory.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
TBR=tommi@webrtc.org
Bug: chromium:611808
Change-Id: Iac59c5b51b950f174119565bac87955a7994bc38
Reviewed-on: https://webrtc-review.googlesource.com/1560
Commit-Queue: Mirko Bonadei <mbonadei@webrtc.org>
Reviewed-by: Henrik Kjellander <kjellander@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#19845}
This CL does not attempt to fix the style issues in the moved tb_ files, at least not yet. In general I've tried to avoid dependencies between the library and vie_auto_test: vie_auto_test depends on the library but not the other way around. I had to make some slight changes to achieve this. I had to remove some ViETest::Log statements in tb_interfaces.cc and I had to move knowledge of where to put output files to the library. I think it ended up being pretty clean in the end but let me know if I missed something. I tried to convert all paths I touched to src-relative paths, so look out if I missed something there.
BUG=
TEST=
Review URL: https://webrtc-codereview.appspot.com/450004
git-svn-id: http://webrtc.googlecode.com/svn/trunk@1923 4adac7df-926f-26a2-2b94-8c16560cd09d
Fixed reproducable random behavior in packet_manipulator.h.
Test is now fully reproducable (runs on only one core) so much tighter limits are now set for the SSIM/PSNR values for the encoding/decoding (verified on all platforms)
BUG=
TEST=out/Debug/video_codecs_test_framework_integrationtests in Debug+Release on Linux, Mac, Windows and in Linux Valgrind.
Review URL: https://webrtc-codereview.appspot.com/381005
git-svn-id: http://webrtc.googlecode.com/svn/trunk@1649 4adac7df-926f-26a2-2b94-8c16560cd09d
Removed old PSNR/SSIM implementations in:
* test/testsupport/metrics/video_metrics.cc
* src/modules/video_coding/codecs/test_framework/test.cc
The functions in video_metrics.cc is now using code in libyuv instead. Old code in test.cc is using the same functions.
The code for video_metrics.h had to be moved into a separate GYP file to avoid circular dependency error on Mac (see issue 160 for more details). The reason for this is that libyuv's unittest target depends on test_support_main.
BUG=
TEST=metrics_unittests in Debug+Release on Linux, Mac and Windows.
Review URL: http://webrtc-codereview.appspot.com/333025
git-svn-id: http://webrtc.googlecode.com/svn/trunk@1325 4adac7df-926f-26a2-2b94-8c16560cd09d
The recorded frame timestamps are used to modify the output video on a frame-per-frame so it can be compared with the reference video using PSNR. This code will make it possible to use vie_auto_test for full stack comparisons with network interference and similar interesting simulations.
There's some refactoring done in vie_comparison_test.cc to make it fit to the new test.
Compiled and executed in Debug+Release on Linux, Mac and Windows.
BUG=
TEST=vie_auto_test --automated --gtest_filter=ViEVideoVerificationTest.*
Review URL: http://webrtc-codereview.appspot.com/320002
git-svn-id: http://webrtc.googlecode.com/svn/trunk@1269 4adac7df-926f-26a2-2b94-8c16560cd09d
Fixed unit tests so they don't use ASSERT_DEATH since that doesn't work with Valgrind.
Fixed all Valgrind warnings except the one caused by CriticalSectionWrapper in system_wrappers.
Reworked all includes and GYP include paths to use full directory paths.
Removed util.h for logging, since it rendered warnings in Valgrind because of gflags. Replaced it with a verbose flag and a new function in video_quality_measurement.cc
BUG=
TEST=Passed test_support_unittests and video_codecs_test_framework_unittests on Linux, Mac and Windows.
Review URL: http://webrtc-codereview.appspot.com/311001
git-svn-id: http://webrtc.googlecode.com/svn/trunk@1126 4adac7df-926f-26a2-2b94-8c16560cd09d
Rewrote the codec test to render to file and do video comparisons.
Refactored the coded tests somewhat. I still need to figure out how to do comparison in the automated case.
Added video analysis to the test. This will make sure that the system output roughly the right thing.
Moved the video metrics library into the test_support library. Made the metrics library available in the automated tests.
Made sure no one passes in too large YUV videos into the autotest.
The standard test's output now gets captured for both the left and right windows.
Wrote a rendering device which just writes the raw frames to file, for analysis. Updated the base standard test to dump its left window output to file. We don't do anything with it yet though.
BUG=
TEST=
Review URL: http://webrtc-codereview.appspot.com/249001
git-svn-id: http://webrtc.googlecode.com/svn/trunk@931 4adac7df-926f-26a2-2b94-8c16560cd09d
The unittest is not ideal for this, but I would have to use similar code as the implementation of the GetOutputDir in order to verify that it actually runs, so it wouldn't make much sense with a test like that.
It compiles and runs on Linux, Win and Mac. The folder gets created and is writeable from other tests.
I have tried using the GetOutputDir from another project that writes output files and it works as intended on all platforms.
Review URL: http://webrtc-codereview.appspot.com/270001
git-svn-id: http://webrtc.googlecode.com/svn/trunk@906 4adac7df-926f-26a2-2b94-8c16560cd09d
The code has no intent to be superportable in all possible scenarios, since it will only be used by our own test code.
I reviewed more sophisticated libraries for doing similar things but came to the conclusion that they introduced more dependencies than motivated for this single purpose.
The unit test has been tested successfully executed on Linux (cmd line and Eclipse), Mac (XCode) and Windows (VS2008).
Review URL: http://webrtc-codereview.appspot.com/223002
git-svn-id: http://webrtc.googlecode.com/svn/trunk@734 4adac7df-926f-26a2-2b94-8c16560cd09d
