using the wrong sample rate for the render signal.
The CL is basically a partial revert of the related changes done on
output_mixer.cc in the CL https://codereview.webrtc.org/1234463003.
The CL also reverts the removal of the input_sample_rate_hz() method
that was removed as part of the CL
https://codereview.webrtc.org/1379123002 (as it was at that point
no longer used).
It should be noted that this CL turns off the effect of the
IntelligibilityEnhancer when the AudioFrame AudioProcessing APIs are
used. While it may be possible to solve that by adding upsampling after
the API call, that approach was discarded due to that:
-That would add extra processing in the echo path, leading to possible
AEC performance reduction.
-That would add extra complexity for the mobile case.
-That would only patch the intelligibility enhancer operation as the
proper way to do such an operation is within APM.
-The intelligibility enhancer is not active by default anywhere.
BUG=webrtc:5237
Review URL: https://codereview.webrtc.org/1525173002
Cr-Commit-Position: refs/heads/master@{#11045}
- Integrates intelligibility into audio_processing.
- Allows modification of reverse stream if intelligibility enabled.
- Makes intelligibility available in audioproc_float test.
- Adds reverse stream processing to audioproc_float.
- (removed) Makes intelligibility toggleable in real time in voe_cmd_test.
- Cleans up intelligibility construction, parameters, constants and dead code.
TBR=pbos@webrtc.org
Review URL: https://codereview.webrtc.org/1234463003
Cr-Commit-Position: refs/heads/master@{#9713}
Reason for revert:
Breaks Chromium FYI content_browsertest on all platforms. The testcase that fails is WebRtcAecDumpBrowserTest.CallWithAecDump.
https://build.chromium.org/p/chromium.webrtc.fyi/builders/Linux/builds/19388
Sample output:
[ RUN ] WebRtcAecDumpBrowserTest.CallWithAecDump
Xlib: extension "RANDR" missing on display ":9".
[4:14:0722/211548:1282124453:WARNING:webrtcvoiceengine.cc(472)] Unexpected codec: ISAC/48000/1 (105)
[4:14:0722/211548:1282124593:WARNING:webrtcvoiceengine.cc(472)] Unexpected codec: PCMU/8000/2 (110)
[4:14:0722/211548:1282124700:WARNING:webrtcvoiceengine.cc(472)] Unexpected codec: PCMA/8000/2 (118)
[4:14:0722/211548:1282124815:WARNING:webrtcvoiceengine.cc(472)] Unexpected codec: G722/8000/2 (119)
[19745:19745:0722/211548:1282133667:INFO:CONSOLE(64)] "Looking at video in element remote-view-1", source: http://127.0.0.1:48819/media/webrtc_test_utilities.js (64)
[19745:19745:0722/211548:1282136892:INFO:CONSOLE(64)] "Looking at video in element remote-view-2", source: http://127.0.0.1:48819/media/webrtc_test_utilities.js (64)
../../content/test/webrtc_content_browsertest_base.cc:62: Failure
Value of: ExecuteScriptAndExtractString( shell()->web_contents(), javascript, &result)
Actual: false
Expected: true
Failed to execute javascript call({video: true, audio: true});.
From javascript: (nothing)
When executing 'call({video: true, audio: true});'
../../content/test/webrtc_content_browsertest_base.cc:75: Failure
Failed
../../content/browser/media/webrtc_aecdump_browsertest.cc:26: Failure
Expected: (base::kNullProcessId) != (*id), actual: 0 vs 0
../../content/browser/media/webrtc_aecdump_browsertest.cc:95: Failure
Value of: GetRenderProcessHostId(&render_process_id)
Actual: false
Expected: true
../../content/browser/media/webrtc_aecdump_browsertest.cc:99: Failure
Value of: base::PathExists(dump_file)
Actual: false
Expected: true
../../content/browser/media/webrtc_aecdump_browsertest.cc:101: Failure
Value of: base::GetFileSize(dump_file, &file_size)
Actual: false
Expected: true
../../content/browser/media/webrtc_aecdump_browsertest.cc:102: Failure
Expected: (file_size) > (0), actual: 0 vs 0
[ FAILED ] WebRtcAecDumpBrowserTest.CallWithAecDump, where TypeParam = and GetParam() = (361 ms)
Original issue's description:
> Allow more than 2 input channels in AudioProcessing.
>
> The number of output channels is constrained to be equal to either 1 or the
> number of input channels.
>
> R=aluebs@webrtc.org, andrew@webrtc.org, pbos@webrtc.org
>
> Committed: c204754b7aTBR=andrew@webrtc.org,aluebs@webrtc.org,ajm@chromium.org,pbos@chromium.org,pbos@webrtc.org,mgraczyk@chromium.org
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
Review URL: https://codereview.webrtc.org/1253573005
Cr-Commit-Position: refs/heads/master@{#9621}
We have two histograms today that trigger on large jumps in either platform reported stream delays (WebRTC.Audio.PlatformReportedStreamDelayJump) or the system delay in the AEC (WebRTC.Audio.AecSystemDelayJump). The latter is the internal buffer size in the AEC.
The sizes of such jumps are of relevance since it can harm the AEC and even put it in a complete failure state. It is hard, not to say impossible, to tell how frequent it is.
Therefore, two complementary histograms are added; number of jumps in each metric.
This way we get a quick way to determine how often a jump occurs in general and also how frequent it is within a call.
This is solved by adding a counter for each metric.
The counter is activated either upon an event trigger or if we know for sure when the AEC is running.
Unfortunately, we can't rely on the destructor at the end of a call so we add a public API for the user to take on the action of calling it at the end of a call.
Tested locally by building ToT chromium including changes and three triggered jumps (200, 50 and 60 ms).
The stats picked up the 60 and 200 ms jumps as expected.
BUG=488124
R=asapersson@webrtc.org, pbos@webrtc.org
Review URL: https://codereview.webrtc.org/1229443003.
Cr-Commit-Position: refs/heads/master@{#9544}
We use this Config struct for enabling/disabling the delay agnostic
AEC. This change renames it to DelayAgnostic for readability reasons.
NOTE: The logic is reversed in this CL. The old ReportedDelay config
turned DA-AEC off, while the new DelayAgnostic turns it on.
The old Config is kept in parallel with the new during a transition
period. This is to avoid problems with API breakages. During this
period, ReportedDelay is disabled or DelayAgnostic is enabled, DA-AEC
is engaged in APM.
BUG=webrtc:4651
R=bjornv@webrtc.org, tommi@webrtc.org
Review URL: https://codereview.webrtc.org/1211053006
Cr-Commit-Position: refs/heads/master@{#9531}
This is a follow-up to r9401, where the configuration DelayCorrection
was replaced by ExtendedFilter.
This change also removes the media constraint
kExperimentalEchoCancellation which was replaced by
kExtendedFilterEchoCancellation in the same CL.
Both settings that are now being removed were kept in the code to avoid
API breakages. In https://codereview.chromium.org/1167343004,
depending code has been updated to avoid breakages.
BUG=webrtc:4696
R=bjornv@webrtc.org, tommi@webrtc.org
Review URL: https://codereview.webrtc.org/1181413004.
Cr-Commit-Position: refs/heads/master@{#9444}
(This reverts commit 3fbf3f8841b5460503fb646eaedcb063620434a8.)
The original submission was reverted because it broke the Chrome build. This is fixed in patch set 2 of this change by keeping the old MediaConstraintsInterface string kExperimentalEchoCancellation. It will be removed once the Chrome code has been updated.
Original description:
"We use this Config struct for enabling/disabling Extended filter mode in AEC. This change renames it to ExtendedFilter for readability reasons. The corresponding media constraint is also renamed to kExtendedFilterEchoCancellation.
The old Config is kept in parallel with the new during a transition period. This is to avoid problems with API breakages. During this period, if any of the two Configs are enabled, the extended filter mode is engaged in APM. That is, the two Configs are combined with an "OR" operation.
This change also renames experimental_aec in AudioOptions to extended_filter_aec."
BUG=webrtc:4696
R=bjornv@webrtc.org, tommi@webrtc.org
Review URL: https://codereview.webrtc.org/1151573021.
Cr-Commit-Position: refs/heads/master@{#9401}
We use this Config struct for enabling/disabling Extended filter mode
in AEC. This change renames it to ExtendedFilter for readability
reasons. The corresponding media constraint is also renamed to
kExtendedFilterEchoCancellation.
The old Config is kept in parallel with the new during a transition
period. This is to avoid problems with API breakages. During this
period, if any of the two Configs are enabled, the extended filter
mode is engaged in APM. That is, the two Configs are combined with an
"OR" operation.
This change also renames experimental_aec in AudioOptions to extended_filter_aec.
BUG=4696
R=bjornv@webrtc.org, tommi@webrtc.org
Review URL: https://webrtc-codereview.appspot.com/54659004
Cr-Commit-Position: refs/heads/master@{#9378}
Because of the Finch experiment, this will not affect Chrome's behaviour at all.
The SNRs in AudioProcessingTest.Formats were only increased to the next multiple of 5.
BUG=webrtc:3146
R=andrew@webrtc.org
Review URL: https://webrtc-codereview.appspot.com/43359004
Cr-Commit-Position: refs/heads/master@{#9263}
The AGC is currently bumping up the mic volume to 33% at startup if it is below that level. This is to avoid getting stuck in a poor state from which the AGC can not move, simply a too low input audio level. For some users, 33% is instead too loud.
This CL gives the user the possibility to set that level at create time.
- Extends the Config ExperimentalAgc with a startup_mic_volume for the user to set if desired. Note that the bump up does not apply to the legacy AGC and the "regular" AGC is controlled by ExperimentalAgc.
- Without any actions, the same default value as previously is used.
- In addition I removed a return value from InitializeExperimentalAgc() and InitializeTransient()
This has been tested by building Chromium on Mac and verify through apprtc that
1) startup_mic_volume = 128 bumps up to 50%.
2) startup_mic_volume = 500 (out of range) bumps up to 100%.
3) startup_mic_volume = 0 bumps up to 4%, the AGC min level.
BUG=4529
TESTED=locally
R=andrew@webrtc.org, kwiberg@webrtc.org
Review URL: https://webrtc-codereview.appspot.com/43109004
Cr-Commit-Position: refs/heads/master@{#9004}
To more easily determine if for example the AEC is not working properly one could monitor how often the estimated delay is out of bounds. With out of bounds we mean either being negative or too large, where both cases will break the AEC.
A new delay metric is added telling the user how often poor delay values were estimated. This is measured in percentage since last time the metrics were calculated.
All APIs have been updated with a third parameter with EchoCancellation::GetDelayMetrics() giving the option to exclude the new metric not to break existing code.
The new metric has been added to audio_processing_unittests with an additional protobuf member, and reference files accordingly updated.
voe_auto_test has not been updated to display the new metric.
BUG=4246
TESTED=audioproc on files
R=aluebs@webrtc.org, andrew@webrtc.org
Review URL: https://webrtc-codereview.appspot.com/39739004
Cr-Commit-Position: refs/heads/master@{#8230}
git-svn-id: http://webrtc.googlecode.com/svn/trunk@8230 4adac7df-926f-26a2-2b94-8c16560cd09d
Take the 50% quantile of the mask and compare it to certain threshold to determine if the desired signal is present. A hold is applied to avoid fast switching between states.
is_signal_present_ has been plotted and looks as expected. The AGC adaptation sounds promising, specially for the cases when the speaker fades in and out from the beam direction.
R=andrew@webrtc.org
Review URL: https://webrtc-codereview.appspot.com/28329005
git-svn-id: http://webrtc.googlecode.com/svn/trunk@8078 4adac7df-926f-26a2-2b94-8c16560cd09d
There are platforms and devices where the reported delays are untrusted and we currently solve that with an extended filter length and a slightly more conservative delay handling.
With this change we give the user the possibility to turn off reported system delay values completely.
- Includes new unit tests.
TESTED=trybots and manual testing
R=aluebs@webrtc.org, kwiberg@webrtc.org
Review URL: https://webrtc-codereview.appspot.com/13629004
git-svn-id: http://webrtc.googlecode.com/svn/trunk@6391 4adac7df-926f-26a2-2b94-8c16560cd09d
Break out the computation to a separate class, and call directly into
this from channel.cc rather than going through AudioProcessing. This
circumvents AudioProcessing's sample rate limitations.
We now compute the RMS over all samples rather than downmixing to a
single channel. This makes the call point in channel.cc easier, is
more "correct" and should have similar (negligible) complexity.
This caused slight changes in the RMS output, so the ApmTest.Process
reference has been updated. Snippet of the failing output:
[ RUN ] ApmTest.Process
Running test 4 of 12...
Value of: rms_level
Actual: 27
Expected: test->rms_level()
Which is: 28
Running test 5 of 12...
Value of: rms_level
Actual: 26
Expected: test->rms_level()
Which is: 27
Running test 6 of 12...
Value of: rms_level
Actual: 26
Expected: test->rms_level()
Which is: 27
Running test 10 of 12...
Value of: rms_level
Actual: 27
Expected: test->rms_level()
Which is: 28
Running test 11 of 12...
Value of: rms_level
Actual: 26
Expected: test->rms_level()
Which is: 27
Running test 12 of 12...
Value of: rms_level
Actual: 26
Expected: test->rms_level()
Which is: 27
BUG=3290
TESTED=Chrome assert is avoided and both voe_cmd_test and apprtc
produce reasonable printed out results from RMS().
R=bjornv@webrtc.org
Review URL: https://webrtc-codereview.appspot.com/16459004
git-svn-id: http://webrtc.googlecode.com/svn/trunk@6056 4adac7df-926f-26a2-2b94-8c16560cd09d
Select "processing" rates based on the input and output sampling rates.
Resample the input streams to those rates, and if necessary to the
output rate.
- Remove deprecated stream format APIs.
- Remove deprecated device sample rate APIs.
- Add a ChannelBuffer class to help manage deinterleaved channels.
- Clean up the splitting filter state.
- Add a unit test which verifies the output against known-working
native format output.
BUG=2894
R=aluebs@webrtc.org, bjornv@webrtc.org, xians@webrtc.org
Review URL: https://webrtc-codereview.appspot.com/9919004
git-svn-id: http://webrtc.googlecode.com/svn/trunk@5959 4adac7df-926f-26a2-2b94-8c16560cd09d
Replace the two versions with a single DownConvertToCodecFormat. As
mentioned in comments, this could be further consolidated with
RemixAndResample but we should write a full audio converter class in
that case.
Along the way:
- Fix the bug present in Channel::Demultiplex with mono input and a
stereo codec.
- Remove the 32 kHz max from the OnDataAvailable path. This avoids a
48 -> 32 -> 48 conversion when VoE is passed 48 kHz audio; instead we
get a straight pass-through to ACM. The 32 kHz conversion is still
needed in the RecordedDataIsAvailable path until APM natively supports
48 kHz.
- Merge resampler improvements from ACM1 to ACM2. This allows ACM to
handle 44.1 kHz audio passed to VoE and was originally done here:
https://webrtc-codereview.appspot.com/1590004
- Reuse the RemixAndResample unit tests for DownConvertToCodecFormat.
- Remove unused functions from utility.cc.
BUG=3155,3000,b/12867572
TESTED=voe_cmd_test using both the OnDataAvailable and
RecordedDataIsAvailable paths, with a captured audio format of all
combinations of {44.1,48} kHz and {1,2} channels, running through all
codecs, and finally using both ACM1 and ACM2.
R=henrika@webrtc.org, turaj@webrtc.org, xians@webrtc.org
Review URL: https://webrtc-codereview.appspot.com/11019005
git-svn-id: http://webrtc.googlecode.com/svn/trunk@5843 4adac7df-926f-26a2-2b94-8c16560cd09d
- Add an Initialize() overload to allow specification of format
parameters. This is mainly useful for testing, but could be used in
the cases where a consumer knows the format before the streams arrive.
- Add a reverse_sample_rate_hz_ parameter to prepare for mismatched
capture and render rates. There is no functional change as it is
currently constrained to match the capture rate.
- Fix a bug in the float dump: we need to use add_ rather than set_.
- Add a debug dump test for both int and float interfaces.
- Enable unpacking of float dumps.
- Enable audioproc to read float dumps.
- Move more shared functionality to test_utils.h, and generally tidy up
a bit by consolidating repeated code.
BUG=2894
TESTED=Verified that the output produced by the float debug dump test is
correct. Processed the resulting debug dump file with audioproc and
ensured that we get identical output. (This is crucial, as we need to
be able to exactly reproduce online results offline.)
R=aluebs@webrtc.org
Review URL: https://webrtc-codereview.appspot.com/9489004
git-svn-id: http://webrtc.googlecode.com/svn/trunk@5676 4adac7df-926f-26a2-2b94-8c16560cd09d
This is mainly to support the native audio format in Chrome. Although
this implementation just moves the float->int conversion under the hood,
we will transition AudioProcessing towards supporting this format
throughout.
- Add a test which verifies we get identical output with the float and
int interfaces.
- The float and int wrappers are tasked with conversion to the
AudioBuffer format. A new shared Process/Analyze method does most of
the work.
- Add a new field to the debug.proto to hold deinterleaved data.
- Add helpers to audio_utils.cc, and start using numeric_limits.
- Note that there was no performance difference between numeric_limits
and a literal value when measured on Linux using gcc or clang.
BUG=2894
R=aluebs@webrtc.org, bjornv@webrtc.org, henrikg@webrtc.org, tommi@webrtc.org, turaj@webrtc.org, xians@webrtc.org
Review URL: https://webrtc-codereview.appspot.com/9179004
git-svn-id: http://webrtc.googlecode.com/svn/trunk@5641 4adac7df-926f-26a2-2b94-8c16560cd09d
