Correction for the calculation of the abs max value
The abs max of a 16 bit integer cannot be represented as a 16 bit integer, because abs(-2^16) is too large. To work around this, we can instead use the index of the max element, convert it to a 32-bit int and then take the absolute value. Bug: chromium:1158070, chromium:1146835, chromium:1161837 Change-Id: If56177c55ec62b4bd578986a5deae38a91bbc821 Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/198123 Commit-Queue: Ivo Creusen <ivoc@webrtc.org> Reviewed-by: Henrik Lundin <henrik.lundin@webrtc.org> Cr-Commit-Position: refs/heads/master@{#32950}
This commit is contained in:
@ -203,7 +203,9 @@ size_t // (o) Estimated lag in end of in[]
|
||||
regressor=in+tlag-1;
|
||||
|
||||
/* scaling */
|
||||
max16 = WebRtcSpl_MaxAbsValueW16(regressor, plc_blockl + 3 - 1);
|
||||
// Note that this is not abs-max, but it doesn't matter since we use only
|
||||
// the square of it.
|
||||
max16 = regressor[WebRtcSpl_MaxAbsIndexW16(regressor, plc_blockl + 3 - 1)];
|
||||
|
||||
const int64_t max_val = plc_blockl * max16 * max16;
|
||||
const int32_t factor = max_val >> 31;
|
||||
|
Reference in New Issue
Block a user