Adds ability to specify desired frame size separate from actual clip
resolution, as well as clip and desired fps.
This allows e.g. reading an HD clip but running benchmarks in VGA, and
to specify e.g. 60fps for the clip but 30for encoding where frame
dropping kicks in so that motion is actually correct rather than just
plaing the clip slowly.
Bug: webrtc:12229
Change-Id: I4ad4fcc335611a449dc2723ffafbec6731e89f55
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/195324
Commit-Queue: Erik Språng <sprang@webrtc.org>
Reviewed-by: Philip Eliasson <philipel@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#32839}
webrtc::test::GetPerfResults() relies on a singleton and this makes
some tests be order dependent (running in a different order makes them
fail).
A good fix is to remove the singleton but this CL at least makes the
fragile test set up the environment correctly.
No-Try: True
Bug: None
Change-Id: I7ad25f685f0bc5d246beeadedfa9f5a39f3547e5
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/197425
Commit-Queue: Mirko Bonadei <mbonadei@webrtc.org>
Reviewed-by: Andrey Logvin <landrey@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#32823}
Calculate quality metrics for dropped frames by comparing original
frame against last decoded one.
This feature makes comparison of encoders which do/don't drop frames
more fair.
The feature is controlled by analyze_quality_of_dropped_frames flag
and is disabled by default.
Bug: none
Change-Id: Ifab8df92d0b76e743ff3193c05d7c8dbd14921c4
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/190660
Commit-Queue: Sergey Silkin <ssilkin@webrtc.org>
Reviewed-by: Sebastian Jansson <srte@webrtc.org>
Reviewed-by: Rasmus Brandt <brandtr@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#32518}
Currently isolated output directory is created in flags_compatibility.py script.
This doesn't work for android swarming tasks because this script isn't called.
Bug: webrtc:11895
Change-Id: I8b8f01850d6e5970292b524d104314eef7ab17be
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/185883
Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org>
Reviewed-by: Artem Titov <titovartem@webrtc.org>
Commit-Queue: Jeremy Leconte <jleconte@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#32236}
When configuring without protobuf this test fails to compile with the error:
perf_test_histogram_writer_no_protobuf.cc:20:1: error: non-void function does not return a value
Bug: None
Change-Id: I8e2676ee4b5284eac08e648fc43bdfc585fc5d64
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/182740
Reviewed-by: Taylor <deadbeef@webrtc.org>
Reviewed-by: Niels Moller <nisse@webrtc.org>
Commit-Queue: Taylor <deadbeef@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#32021}
Since the flag is now on by default, we can remove it (after all
callers stop passing it).
We can also remove all Chart JSON code from WebRTC since it is
no longer used.
Requires one recipe CL and one downstream CL to land first.
Bug: chromium:1029452
Change-Id: Ic1d62e8ab9dfcd255cd2bf51d153db80d59c564b
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/171878
Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org>
Commit-Queue: Patrik Höglund <phoglund@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30927}
The proto code is copied into the out dir, so always use that since
it is what isolate is using. Previously we pointed straight at the
checkout code.
I think copying python into the out dir is probably the right way
to do things, so we should go that way in the future.
Bug: chromium:1029452
Change-Id: I701cc84a674021d2f78c73db8808f55cd6ae5174
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/171877
Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org>
Commit-Queue: Patrik Höglund <phoglund@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30923}
This reverts commit 7b201012bc7f7cd95249b8314d0d7ebabe966d8b.
Reason for revert: Seems to work, but need to get low bw tests working first
Original change's description:
> Flip histograms to true by default, fix unit in isac_fix_test.
>
> Requires downstream changes for all WebRTC perf tests, and
> a corresponding recipe change so isac_fix_test starts using the new
> flow.
>
> Bug: chromium:1029452
> Change-Id: I8918fca9bef003d365037c1c6bf7c55747dfed99
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/170633
> Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org>
> Commit-Queue: Patrik Höglund <phoglund@webrtc.org>
> Cr-Commit-Position: refs/heads/master@{#30906}
TBR=phoglund@webrtc.org,mbonadei@webrtc.org
Change-Id: I96c2309cd71be14c5a27b515736a32f1b256453c
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: chromium:1029452
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/171865
Reviewed-by: Patrik Höglund <phoglund@webrtc.org>
Commit-Queue: Patrik Höglund <phoglund@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30913}
Requires downstream changes for all WebRTC perf tests, and
a corresponding recipe change so isac_fix_test starts using the new
flow.
Bug: chromium:1029452
Change-Id: I8918fca9bef003d365037c1c6bf7c55747dfed99
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/170633
Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org>
Commit-Queue: Patrik Höglund <phoglund@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30906}
It's better to set this to the right value in the C++ API rather
than the hack in catapult_uploader.py.
Bug: chromium:1029452
Change-Id: Ia942ff22f8422874cd226e6a7fdce20333ac4a50
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/170632
Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org>
Commit-Queue: Patrik Höglund <phoglund@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30817}
This is a reland of 30026214b13535a9fe1c47f1463378fbf976c643
Original change's description:
> Fix for out-of-bounds write in square test frame generator.
>
> The length is set on construction and includes an assumption on the
> image resolution, if the resolution changes, a square might be larger
> than what fits into the buffer, causing an out of bounds write. This
> CL fixes this simply by restricting the size of the square.
>
> Bug: webrtc:11415
> Change-Id: Iee14a1971997b4ae2fddef0a7af7c76a2509e879
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/170042
> Commit-Queue: Sebastian Jansson <srte@webrtc.org>
> Reviewed-by: Ali Tofigh <alito@webrtc.org>
> Cr-Commit-Position: refs/heads/master@{#30732}
Bug: webrtc:11415
Change-Id: I0dc584858208f478434ebc6f9e31634595c4e5ad
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/170116
Reviewed-by: Ali Tofigh <alito@webrtc.org>
Commit-Queue: Sebastian Jansson <srte@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30779}
We need to write protos as "wb" and not "w", otherwise we get CRLF
on Windows which corrupts the proto.
Bug: chromium:1029452
Change-Id: Iabf841405134d7bc2523ac48219ca7cb9d8214c1
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/170320
Commit-Queue: Patrik Höglund <phoglund@webrtc.org>
Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30772}
This reverts commit 30026214b13535a9fe1c47f1463378fbf976c643.
Reason for revert: Speculative revert, breaks downstream test.
Original change's description:
> Fix for out-of-bounds write in square test frame generator.
>
> The length is set on construction and includes an assumption on the
> image resolution, if the resolution changes, a square might be larger
> than what fits into the buffer, causing an out of bounds write. This
> CL fixes this simply by restricting the size of the square.
>
> Bug: webrtc:11415
> Change-Id: Iee14a1971997b4ae2fddef0a7af7c76a2509e879
> Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/170042
> Commit-Queue: Sebastian Jansson <srte@webrtc.org>
> Reviewed-by: Ali Tofigh <alito@webrtc.org>
> Cr-Commit-Position: refs/heads/master@{#30732}
TBR=srte@webrtc.org,alito@webrtc.org
Change-Id: Ia0056da04a6f6f817ccadfc38aabe0c5f94754cc
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: webrtc:11415
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/170115
Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org>
Commit-Queue: Mirko Bonadei <mbonadei@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30743}
The length is set on construction and includes an assumption on the
image resolution, if the resolution changes, a square might be larger
than what fits into the buffer, causing an out of bounds write. This
CL fixes this simply by restricting the size of the square.
Bug: webrtc:11415
Change-Id: Iee14a1971997b4ae2fddef0a7af7c76a2509e879
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/170042
Commit-Queue: Sebastian Jansson <srte@webrtc.org>
Reviewed-by: Ali Tofigh <alito@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30732}
This is the WebRTC equivalent of testing/perf/perf_result_reporter.h
in Chromium. That class was introduced because the PrintResult
functions are quite hard to use right. It was easy to mix up
metrics, modifiers and stories, for instance.
I choose to introduce this new class because I need to create a new
API for PrintResult anyway. For instance, the important bool isn't
really supported by histograms. Also I would like to restrict units
to an enum because you cannot make up your own units anymore.
We could also have had a strictly checked string type, but that's
bad API design. An enum is better because the compiler will check
that the unit is valid rather than at runtime.
Furthermore, down the line we can probably make each reporter write
protos directly to /tmp and merge them later, instead of having a
singleton which writes results at the end and keeps all test results
in memory. This abstraction makes it easy to make a clean and simple
implementation of just that.
Steps:
1) land this
2) start rewriting perf tests to use this class
3) nuke PrintResult functions
4) don't convert units to string, convert directly from Unit
to proto::Unit
5) write protos directly from this class (either through
a singleton or directly) and nuke the perf results writer
abstraction.
Bug: chromium:1029452
Change-Id: Ia919c371a69309130c797fdf01ae5bd64345ab2e
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/168770
Reviewed-by: Artem Titov <titovartem@webrtc.org>
Commit-Queue: Patrik Höglund <phoglund@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30599}
I think these functions are so hard to understand, so I tried to
make an as grounded example as possible.
Bug: chromium:1029452
Change-Id: I5d4284bc15b39cb94ba42d2c483a619ecf42fb91
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/168945
Reviewed-by: Artem Titov <titovartem@webrtc.org>
Commit-Queue: Patrik Höglund <phoglund@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30590}
This will be used by WebRTC tests. It converts results exactly the
same as our downstream implementation (histogram_util).
This implementation should be pretty feature complete, or at least
enough to start testing the end-to-end flow. I will set up some
experimental recipe code and see if this actually makes it into the
dashboard.
Note: needs some catapult changes to land first and be rolled
into Chromium, and then WebRTC.
Bug: chromium:1029452
Change-Id: I939046929652fc27b8fcb18af54bde22886d9228
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/166172
Commit-Queue: Patrik Höglund <phoglund@webrtc.org>
Reviewed-by: Artem Titov <titovartem@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30436}
This CL was generated by running:
git ls-files | grep ".cc" | xargs perl -i -ne 'BEGIN {undef $/}; s/("[\s\n]*<<[\s\n]*")/" "/g; print;'; git cl format
After that I manually edited modules/audio_processing/gain_controller2.cc to preserve its original
formatting.
This primary benefit of this change is a small reduction in binary size.
Bug: None
Change-Id: I689fa7ba9c717c314bb167e5d592c3c4e0871e29
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/165961
Reviewed-by: Alessio Bazzica <alessiob@webrtc.org>
Reviewed-by: Karl Wiberg <kwiberg@webrtc.org>
Commit-Queue: Jonas Olsson <jonasolsson@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30251}
The new interface is called PerfTestResultWriter and is currently
implemented by PerfResultsLogger (renamed PerfTestGraphJsonWriter).
I plan to introduce a second implementation of the perf logger that
uses the new Histogram C++ API. I add a flag that chooses
between the two implementations so I can try it out (perhaps by
setting up a second, limited run of webrtc_perf_tests on the perf
bots that uses the new implementation). The histogram C++
implementation will come in the next patch.
As a side effect, I disentangled the plottable counter printer from
the perf result printer so it will work for both implementations.
The only thing they had in common was that both wrote JSON anyway.
See the bug for details on the new API.
Bug: chromium:1029452
Change-Id: Icb21b25ced08ea73aeecd221e9d51f2adf3dab1b
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/165389
Reviewed-by: Artem Titov <titovartem@webrtc.org>
Commit-Queue: Patrik Höglund <phoglund@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30243}
This will make RESULT lines still come out after we add a second JSON
writer implementation.
Bug: chromium:1029452
Change-Id: I5cba3151c21df2901f19305e9b71bc5c9638a0ae
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/165399
Reviewed-by: Artem Titov <titovartem@webrtc.org>
Commit-Queue: Patrik Höglund <phoglund@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30208}
Split out of https://webrtc-review.googlesource.com/c/src/+/165389.
I disentangled the plottable counter printer from the perf result
printer so it will work for both future implementations of the perf
test JSON writers. The only thing plottable counters and the
results writer had in common was that both wrote JSON anyway.
Bug: chromium:1029452
Change-Id: I041c3096641eda42542e8d994b246eb313940b4b
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/165397
Commit-Queue: Patrik Höglund <phoglund@webrtc.org>
Reviewed-by: Artem Titov <titovartem@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30198}
It was found that generator can be destroyed on another thread
comparing to the one, from which frame were generated. It can happen
because generator injected into PC though scoped_ref object and the
last pointer to that object can be destroyed on different thread
depending on machine load.
To fix this sequence checker is replaced with lock. It is required
to ensure that generator won't be destroyed while it is reading frame,
because otherwise it can catch SIGSEGV.
Bug: webrtc:10138
Change-Id: Ia3488bd8ae396c209b90977469593784bb82114b
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/162182
Commit-Queue: Artem Titov <titovartem@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#30086}
Or, well, to be fair it still kind of does the same thing, but
the thing it's (void)ing in is a lot more related to what it
actually happening. I could not find another way to solve this
since fileutils is fundamentally optional to unit tests, but the
flag isn't.
Bug: webrtc:9792
Change-Id: I6ebf012246bc259883bc0aaf73ac7fea5525dd1f
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/157101
Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org>
Commit-Queue: Patrik Höglund <phoglund@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#29501}
I had to change approach. Unfortunately we can't expect
that test_main_lib users link with fileutils, which causes it to
not link when the override symbol is missing.
New approach: resources_dir_flag is now a separate target, it
will be depended upon by the downstream override, which just
reads the flag and returns it as the resource dir. This gets
rid of the mutable state downstream as well.
So:
1) Land this
2) Make downstream read the flag instead of keeping its own state
3) Remove OverrideResourcesDir upstream and clean up the hacks
4) Remove the now orphaned OverrideResourcesDir downstream
Bug: webrtc:9792
Change-Id: Ic2ef3910bb5d39d9fb71e06fbbbb6aec4de52e78
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/157041
Commit-Queue: Patrik Höglund <phoglund@webrtc.org>
Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#29487}
We needed a hack in test_main_lib.cc to ensure fileutils were always
linked with test binaries downstream. When I removed the hack, it
broke the binaries that were _not_ using fileutils because a certain
bazel rule expects to be able to pass the flag to all test binaries.
The solution is to move the flag to test_main_lib.cc. This is the
right place for it since it's apparently in the contract of a WebRTC
test binary to support this flag. We then have to pass the value
down to the override, which is why I add a new function for that.
I leave the flag unimplemented in OSS because no one is using it
here anyway. It will be implemented downstream.
Bug: webrtc:9792
Change-Id: I21b3deb43bf0cd56d6aa2622dc5519370a0307a9
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/156568
Reviewed-by: Mirko Bonadei <mbonadei@webrtc.org>
Commit-Queue: Patrik Höglund <phoglund@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#29474}
