zqz/platform-external-webrtc
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Support e.g. --workers=2x to use two workers per core.

Browse Source 
This is mostly useful for tests performing a lot of I/O and sleeping,
when you don't know on which architecture they end up running.

The syntax can also be used to reduce CPU load (e.g. --workers=0.5x).

Bug: webrtc:9717
Change-Id: I26b4552576b1dd56a69c2223da39f4bb1115bbf6
Reviewed-on: https://webrtc-review.googlesource.com/101643
Commit-Queue: Yves Gerey <yvesg@webrtc.org>
Reviewed-by: Patrik Höglund <phoglund@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#24830}
This commit is contained in:
Yves Gerey
2018-09-25 15:34:27 +02:00
committed by Commit Bot
parent a3b9b27412
commit ea766fa7a2
2 changed files with 91 additions and 30 deletions
Download Patch File Download Diff File Expand all files Collapse all files

19
tools_webrtc/gtest-parallel-wrapper.py
View File

@ -15,8 +15,9 @@ gtest-parallel, renaming options and translating environment variables into
flags. Developers should execute gtest-parallel directly.
In particular, this translates the GTEST_SHARD_INDEX and GTEST_TOTAL_SHARDS
environment variables to the --shard_index and --shard_count flags, and renames
the --isolated-script-test-output flag to --dump_json_test_results.
environment variables to the --shard_index and --shard_count flags, renames
the --isolated-script-test-output flag to --dump_json_test_results,
and interprets e.g. --workers=2x as 2 workers per core.
Flags before '--' will be attempted to be understood as arguments to
gtest-parallel. If gtest-parallel doesn't recognize the flag or the flag is
@ -63,6 +64,7 @@ Will be converted into:
import argparse
import collections
import multiprocessing
import os
import shutil
import subprocess
@ -81,6 +83,15 @@ def _CatFiles(file_list, output_file):
output_file.write(input_file.read())
os.remove(filename)
def _ParseWorkersOption(workers):
"""Interpret Nx syntax as N * cpu_count. Int value is left as is."""
base = float(workers.rstrip('x'))
if workers.endswith('x'):
result = int(base * multiprocessing.cpu_count())
else:
result = int(base)
return max(result, 1) # Sanitize when using e.g. '0.5x'.
class ReconstructibleArgumentGroup(object):
"""An argument group that can be converted back into a command line.
@ -117,13 +128,15 @@ def ParseArgs(argv=None):
gtest_group.AddArgument('-d', '--output_dir')
gtest_group.AddArgument('-r', '--repeat')
gtest_group.AddArgument('--retry_failed')
gtest_group.AddArgument('-w', '--workers')
gtest_group.AddArgument('--gtest_color')
gtest_group.AddArgument('--gtest_filter')
gtest_group.AddArgument('--gtest_also_run_disabled_tests',
action='store_true', default=None)
gtest_group.AddArgument('--timeout')
# Syntax 'Nx' will be interpreted as N * number of cpu cores.
gtest_group.AddArgument('-w', '--workers', type=_ParseWorkersOption)
# --isolated-script-test-output is used to upload results to the flakiness
# dashboard. This translation is made because gtest-parallel expects the flag
# to be called --dump_json_test_results instead.