Correction for the calculation of the abs max value

The abs max of a 16 bit integer cannot be represented as a 16 bit integer, because abs(-2^16) is too large. To work around this, we can instead use the index of the max element, convert it to a 32-bit int and then take the absolute value.

Bug: chromium:1158070, chromium:1146835, chromium:1161837
Change-Id: If56177c55ec62b4bd578986a5deae38a91bbc821
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/198123
Commit-Queue: Ivo Creusen <ivoc@webrtc.org>
Reviewed-by: Henrik Lundin <henrik.lundin@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#32950}
This commit is contained in:
Ivo Creusen
2021-01-12 16:01:30 +01:00
committed by Commit Bot
parent f23e2144e8
commit fe06dbdfa2
2 changed files with 7 additions and 4 deletions

View File

@ -203,7 +203,9 @@ size_t // (o) Estimated lag in end of in[]
regressor=in+tlag-1;
/* scaling */
max16 = WebRtcSpl_MaxAbsValueW16(regressor, plc_blockl + 3 - 1);
// Note that this is not abs-max, but it doesn't matter since we use only
// the square of it.
max16 = regressor[WebRtcSpl_MaxAbsIndexW16(regressor, plc_blockl + 3 - 1)];
const int64_t max_val = plc_blockl * max16 * max16;
const int32_t factor = max_val >> 31;