Commit Graph

4101 Commits

Author SHA1 Message Date
348e411dd2 Adds data logging in native AudioDeviceBuffer class.
Goal is to provide periodic logging of most essential audio parameters
for playout and recording sides. It will allow us to track if the native audio layer is working as intended.

BUG=NONE

Review-Url: https://codereview.webrtc.org/2132613002
Cr-Commit-Position: refs/heads/master@{#13440}
2016-07-12 09:18:46 +00:00
11dd52bcc6 H264VideoToolboxEncoder: Add support for native handles
Support encoding from CVPixelBuffers directly in H264VideoToolboxEncoder.
If the frame needs to be scaled, it will fall back to the previous slow
path:
CVPixelBuffer -> NV12 -> I420 -> (scale) I420 -> NV12 -> CVPixelBuffer.

BUG=webrtc:4081

Review-Url: https://codereview.webrtc.org/2140573002
Cr-Commit-Position: refs/heads/master@{#13439}
2016-07-12 08:26:47 +00:00
b4d31085b4 Added various timestamps to FrameObject.
Added various timestamps to the FrameObject class which are needed to calculate
the jitter delay.

BUG=webrtc:5514

Review-Url: https://codereview.webrtc.org/2124943002
Cr-Commit-Position: refs/heads/master@{#13434}
2016-07-11 15:46:35 +00:00
c2c24f795e Remove frame rate and target bitrate from ProtectionBitrateCalculator::SetEncodingData
These arguments are not really known when calling SetEncodingData. They are still provided as argument to ProtectionBitrateCalculator::SetTargetRates though.

This cl is broken out from https://codereview.webrtc.org/2060403002/
BUG=webrtc:5687

Review-Url: https://codereview.webrtc.org/2121983002
Cr-Commit-Position: refs/heads/master@{#13429}
2016-07-11 08:47:36 +00:00
5e12d36ba7 Reset InterArrival if arrival time clock makes a jump.
Also adds a copy of the BWE test suite to the new DelayBasedBwe class.

BUG=webrtc:6079

Review-Url: https://codereview.webrtc.org/2126793002
Cr-Commit-Position: refs/heads/master@{#13428}
2016-07-11 08:44:12 +00:00
a49f1105eb Revert of Reland Issue 2061423003: Refactor NACK bitrate allocation (patchset #1 id:1 of https://codereview.webrtc.org/2131313002/ )
Reason for revert:
It keeps breaking upstream.

Original issue's description:
> Reland Issue 2061423003: Refactor NACK bitrate allocation
>
> This is a reland of https://codereview.webrtc.org/2061423003/
> Which was reverted in https://codereview.webrtc.org/2131913003/
>
> The reason for the revert was that some upstream code used
> RtpSender::SetTargetBitrate(). I've added that back as a no-op until we
> it's been brought up to date.
>
> TBR=tommi@webrtc.org
>
> Committed: 05ce4ae31f

TBR=tommi@webrtc.org,sprang@webrtc.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true

Review-Url: https://codereview.webrtc.org/2130423002
Cr-Commit-Position: refs/heads/master@{#13419}
2016-07-08 18:02:02 +00:00
05ce4ae31f Reland Issue 2061423003: Refactor NACK bitrate allocation
This is a reland of https://codereview.webrtc.org/2061423003/
Which was reverted in https://codereview.webrtc.org/2131913003/

The reason for the revert was that some upstream code used
RtpSender::SetTargetBitrate(). I've added that back as a no-op until we
it's been brought up to date.

TBR=tommi@webrtc.org

Review URL: https://codereview.webrtc.org/2131313002 .

Cr-Commit-Position: refs/heads/master@{#13418}
2016-07-08 17:11:23 +00:00
e5dd44101e Revert of Refactor NACK bitrate allocation (patchset #16 id:300001 of https://codereview.webrtc.org/2061423003/ )
Reason for revert:
Breaks upstream code.

Original issue's description:
> Refactor NACK bitrate allocation
>
> Nack bitrate allocation should not be done on a per-rtp-module basis,
> but rather shared bitrate pool per call. This CL moves allocation to the
> pacer and cleans up a bunch if bitrate stats handling.
>
> BUG=
> R=danilchap@webrtc.org, stefan@webrtc.org, tommi@webrtc.org
>
> Committed: 5fc59e810b

TBR=tommi@webrtc.org,danilchap@webrtc.org,stefan@webrtc.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG=

Review-Url: https://codereview.webrtc.org/2131913003
Cr-Commit-Position: refs/heads/master@{#13417}
2016-07-08 16:39:02 +00:00
5fc59e810b Refactor NACK bitrate allocation
Nack bitrate allocation should not be done on a per-rtp-module basis,
but rather shared bitrate pool per call. This CL moves allocation to the
pacer and cleans up a bunch if bitrate stats handling.

BUG=
R=danilchap@webrtc.org, stefan@webrtc.org, tommi@webrtc.org

Review URL: https://codereview.webrtc.org/2061423003 .

Cr-Commit-Position: refs/heads/master@{#13416}
2016-07-08 16:15:29 +00:00
108ecec51c Removed LEGACY_BITEXACT from neteq_impl.cc and updated the ACM unit tests.
I'll be rewriting AcmReceiver soon and am trying to reduce the amount of
old stuff that needs to be supported.

I've manually checked the outputs of the AcmReceiver bitexactness
tests with this change. A large part of the tests are still bitexact,
with one section only differing slightly in timings. Nothing audible
unless playing the old and new versions back simultaneously.

The output of NetEqDecoderTest were also changed due to this CL, although only on android. I built and ran the test locally and compared the audio output manually - the changes were the same as for the other tests; i.e. very slight timing changes for a part of the output.

I updated the network stats checksum for android without analyzing it further. I expect it goes hand-in-hand with the changes to the output; i.e. the changes in it are fine because the audio output is fine. Likely, the stats will show changes in the usage of CNG, since that is what the code changes.

BUG=webrtc:1361

Review-Url: https://codereview.webrtc.org/2117763002
Cr-Commit-Position: refs/heads/master@{#13415}
2016-07-08 15:45:26 +00:00
ae4f7674e4 Style updates for ForwardErrorCorrection and related classes.
This CL mainly updates the FEC code to use more C++11 features and
to be more in line with the style guide. These changes should
have no impact on the functionality provided by the FEC.

Summary of style fixes:
- Use range-based for loops, where applicable.
- Use auto type deduction for iterator type names.
- Use RTC_DCHECK instead of assert.
- Rename FEC to Fec, where applicable.
- Update test_fec.cc to use variable_names rather than variableNames.
- Avoid redefining the PacketList types outside ForwardErrorCorrection.

Another minor change is that storage for the packet masks, as these
are generated, now is provided by a member variable, rather than
being dynamically allocated on every call to GenerateFec.

BUG=webrtc:5654
R=danilchap@webrtc.org, stefan@webrtc.org

Review URL: https://codereview.webrtc.org/2080553003 .

Cr-Commit-Position: refs/heads/master@{#13403}
2016-07-07 07:41:03 +00:00
9b522f8d04 Add more logging about the bwe state and VideoSendStream state.
Review-Url: https://codereview.webrtc.org/2121273003
Cr-Commit-Position: refs/heads/master@{#13402}
2016-07-07 07:36:40 +00:00
63fb95a68d Fixed time moving backwards in the AudioCodingModule.
There was a fast path in PreprocessToAddData that would just use the
input timestamps if the input format was equal to the required format of
the encoder. This works well as long as the codec never changes. If we
are first doing resampling (specifically upsampling) and then change to
a codec that does not require resampling, we'll need to stick to
whatever input timestamp we left off at, rather than silently accepting
whatever we're sent.

BUG=622435

Review-Url: https://codereview.webrtc.org/2119393002
Cr-Commit-Position: refs/heads/master@{#13398}
2016-07-06 16:34:31 +00:00
8513029e1f Remove old WebRTC-NewVideoJitterBuffer used for testing the NackModule.
This experiment was used to test the NackModule but will soon (tm) be used to
test the completly new video jitter buffer.

BUG=webrtc:5514
R=stefan@webrtc.org

Review URL: https://codereview.webrtc.org/2123913002 .

Cr-Commit-Position: refs/heads/master@{#13395}
2016-07-06 14:10:45 +00:00
3f33e2aa54 Minor refactoring of the AudioDeviceBuffer class
BUG=NONE
NOTRY=true

Review-Url: https://codereview.webrtc.org/2117303002
Cr-Commit-Position: refs/heads/master@{#13389}
2016-07-06 07:34:02 +00:00
fa1d568730 Added NiceMock for MockRtcEventLog in several unittests where we don't care about the event logging.
BUG=webrtc:4741
R=stefan@webrtc.org

Review URL: https://codereview.webrtc.org/2121993002 .

Cr-Commit-Position: refs/heads/master@{#13388}
2016-07-06 07:12:21 +00:00
f5b2e519b4 Fix stats for encoder target bitrate when target rate is zero.
When the target bitrate is zero, currently  VideoSendStream.Stats.target_media_bitrate_bps show the last set rate before the target was set to zero.

BUG=webrtc::5687 b/29574845

Review-Url: https://codereview.webrtc.org/2122743003
Cr-Commit-Position: refs/heads/master@{#13386}
2016-07-05 15:34:08 +00:00
9b2ce6be09 Padding is now used to check for continuity in the packet sequence.
BUG=webrtc:5514

Review-Url: https://codereview.webrtc.org/2051453002
Cr-Commit-Position: refs/heads/master@{#13383}
2016-07-05 12:04:52 +00:00
956ed71a11 TransportFeedback must be able to start with dropped packets.
A bug in the transpot feedback adapter causes new feedback message to
always start with a received packet. This makes it impossible for the
receiver to distinguish from actual dropped packets and dropped feedback
messages.

BUG=webrtc:6073
R=stefan@webrtc.org

Review URL: https://codereview.webrtc.org/2122863002 .

Cr-Commit-Position: refs/heads/master@{#13381}
2016-07-05 10:01:08 +00:00
552866c402 FrameObject inherit from VCMEncodedFrame.
Let the FrameObject class inherit from VCMEncodedFrame since the rest of the
decoding pipeline use VCMEncodedFrame.

BUG=webrtc:5514
R=stefan@webrtc.org

Review URL: https://codereview.webrtc.org/2110543005 .

Cr-Commit-Position: refs/heads/master@{#13380}
2016-07-04 14:19:11 +00:00
14d5dbe5b3 Reland of "Move RtcEventLog object from inside VoiceEngine to Call.", "Fix to make the start/stop functions for the Rtc Eventlog non-virtual." and "Fix for RtcEventLog ObjC interface"
The breaking tests in Chromium have been temporarily disabled, they will be fixed and reenabled soon.

Original CLs: https://codereview.webrtc.org/1748403002/, https://codereview.webrtc.org/2107253002/ and https://codereview.webrtc.org/2106103003/.

TBR=solenberg@webrtc.org,tommi@webrtc.org,stefan@webrtc.org,terelius@webrtc.org,tkchin@webrtc.org
BUG=webrtc:4741, webrtc:5603, chromium:609749

Review-Url: https://codereview.webrtc.org/2110113003
Cr-Commit-Position: refs/heads/master@{#13379}
2016-07-04 14:07:03 +00:00
77ad394fa6 A simple copy of the old audio mixer to a new directory.
I have added build files and renamed the mixer so that it doesn't conflict with the old one. The header includes now point to this copy of the mixer. I have also fixed some of the more obvious cases of style guide non-conformance and run 'PRESUBMIT' on the old mixer.

This is a first step in the creation of a new mixing module that will replace AudioConferencMixer and OutputMixer.

NOTRY=True

Review-Url: https://codereview.webrtc.org/2104363003
Cr-Commit-Position: refs/heads/master@{#13378}
2016-07-04 13:33:09 +00:00
0fd6801c3c clang-format on AudioDeviceBuffer
BUG=NONE
R=magjed@webrtc.org

Review URL: https://codereview.webrtc.org/2119093003 .

Cr-Commit-Position: refs/heads/master@{#13377}
2016-07-04 11:01:41 +00:00
414dda1a10 Change VCMFrameBuffer and VCMEncodedFrame to use rotation from base class EncodedImage.
BUG=webrtc:5687

Review-Url: https://codereview.webrtc.org/2037633002
Cr-Commit-Position: refs/heads/master@{#13376}
2016-07-04 08:45:28 +00:00
2c332bb682 Simplify logging statements.
BUG=NONE
R=henrika@webrtc.org

Review URL: https://codereview.webrtc.org/2115603004 .

Cr-Commit-Position: refs/heads/master@{#13375}
2016-07-04 07:03:54 +00:00
bb01781982 Adding back removed methods to MockNonlinearBeamformer.
This is temporary, until downstream dependencies are updated.

TBR=aluebs@webrtc.org

Review-Url: https://codereview.webrtc.org/2121553002
Cr-Commit-Position: refs/heads/master@{#13374}
2016-07-02 20:34:56 +00:00
f4022ffa1a Pull out the PostFilter to its own NonlinearBeamformer API
This is done to avoid having a nonlinear component in the AEC path.
Now the linear delay and sum is run before the AEC and the postfilter after it.

This change landed originally at: https://codereview.webrtc.org/1982183002/

R=peah@webrtc.org
TBR=henrik.lundin@webrtc.org

Review URL: https://codereview.webrtc.org/2110593003 .

Cr-Commit-Position: refs/heads/master@{#13371}
2016-07-02 00:19:32 +00:00
1aa821980d Add logging to Intelligibility Enhancer
It logs when the IE is activated and deactivated.

R=ivoc@webrtc.org, turaj@webrtc.org

Review URL: https://codereview.webrtc.org/2104273002 .

Cr-Commit-Position: refs/heads/master@{#13370}
2016-07-02 00:16:13 +00:00
fe3654d5dc Add native handle support to SimulcastEncoderAdapter.
If all subencoders support textures, the adapter will claim support.
Texture frames will be passed on directly to subencoders, without any
attempt at scaling, and subencoders will be expected to sample/scale
correctly from source textures.

BUG=
NOTRY=true

Review-Url: https://codereview.webrtc.org/2099483002
Cr-Commit-Position: refs/heads/master@{#13365}
2016-07-01 16:06:00 +00:00
e703f41336 Remove unused copy of last received keyframe.
Copy method was the user of those fields but it has been removed in May 2015 by refs/heads/master@{#9244}.

R=pbos@webrtc.org

Review URL: https://codereview.webrtc.org/2113973002 .

Cr-Commit-Position: refs/heads/master@{#13362}
2016-07-01 11:36:21 +00:00
84cab205f5 UMA log for audio_device Init and Start(Playout|Recording). Make Init return a more specific error code, if possible.
BUG=webrtc:5761
R=asapersson@webrtc.org, henrika@webrtc.org

Review URL: https://codereview.webrtc.org/2103863004 .

Cr-Commit-Position: refs/heads/master@{#13361}
2016-07-01 11:35:31 +00:00
02bafc6379 Add a race-checking mechanism.
Permits CHECKing/DCHECKing that methods are being accessed in a
thread-safe manner, even if they are not used by one single thread
(thread pools such as VideoToolbox OK).

BUG=
R=danilchap@webrtc.org, tommi@webrtc.org

Review URL: https://codereview.webrtc.org/2097403002 .

Cr-Commit-Position: refs/heads/master@{#13358}
2016-07-01 10:45:29 +00:00
e15032c750 Remove all old suspension logic.
I'm also removing media_optimization_unittest.cc, since it only tested the
suspension logic and nothing else.

R=pbos@webrtc.org

Review URL: https://codereview.webrtc.org/2119503002 .

Cr-Commit-Position: refs/heads/master@{#13355}
2016-07-01 07:00:19 +00:00
5041110b94 Compensate for the LevelController gain in the IntelligibilityEnhancer
R=peah@webrtc.org

Review URL: https://codereview.webrtc.org/2112553003 .

Cr-Commit-Position: refs/heads/master@{#13353}
2016-06-30 22:35:46 +00:00
a181c9ad17 Keep track of the user-facing number of channels in a ChannelBuffer
Before this change the ChannelBuffer had a fixed number of channels. This meant for example that when the Beamformer would reduce the number of channels to one, the merging filter bank was still merging all the channels, which was unnecessary since they were not processed and just discarded later. This change doesn't change the signal at all. It just reflects the number of channels in the ChannelBuffer, reducing the complexity.

R=henrik.lundin@webrtc.org, peah@webrtc.org, tina.legrand@webrtc.org

Review URL: https://codereview.webrtc.org/2053773002 .

Cr-Commit-Position: refs/heads/master@{#13352}
2016-06-30 22:33:47 +00:00
b59ff8952f This CL provides improved parameter tuning for the level controller as well as some further minor changes.
It does:
-Handle saturations in a better manner by adding different gain change
step sizes for upwards and downwards changes, as well as when there
is saturation.
-Handle conditions with initial noise-only regions in a better way by
setting a high initial peak level estimate which is gradually reduced until
certainty about the peak level is achieved.
-Limit the maximum gain to limit noise amplification, and to reflect that it
initially is intended to be used in cascade with the fixed digital AGC mode.
-Lower the maximum allowed stationary noise floor to reduce the risk of
excessive noise amplification.
-Lower the target gain to reduce the risk of causing the AEC on the other
end to fail due to high playout levels triggering nonlinearities.
This also reduces the risk for saturation.
-Handle the noise-only regions in a better manner.

NOTRY=true
TBR=aleloi
BUG=webrtc:5920

Review-Url: https://codereview.webrtc.org/2111553002
Cr-Commit-Position: refs/heads/master@{#13350}
2016-06-30 16:19:41 +00:00
504c47d750 FrameBuffer2 now has Start/Stop methods.
The Stop method is used to signal any thread that is waiting in the
NextFrame function and will cause it to return immediately.

BUG=webrtc:5514
R=pbos@webrtc.org

Review URL: https://codereview.webrtc.org/2105323002 .

Cr-Commit-Position: refs/heads/master@{#13349}
2016-06-30 15:33:17 +00:00
5f6547e89c Disabling the performance unittests on Android for the level controller.
NOTRY=true
BUG=

Review-Url: https://codereview.webrtc.org/2108723004
Cr-Commit-Position: refs/heads/master@{#13346}
2016-06-30 12:13:46 +00:00
03dd6db5cb Added empty directory with myself as owner for new mixer.
NOTRY=True

Review-Url: https://codereview.webrtc.org/2109133003
Cr-Commit-Position: refs/heads/master@{#13345}
2016-06-30 12:03:17 +00:00
9e03c3b372 Revert of Move RtcEventLog object from inside VoiceEngine to Call. (patchset #16 id:420001 of https://codereview.webrtc.org/1748403002/ )
Reason for revert:
Reverting all CLs related to moving the eventlog, as they break Chromium tests.

Original issue's description:
> Move RtcEventLog object from inside VoiceEngine to Call.
>
> In addition to moving the logging object itself, this also moves the interface from PeerConnectionFactory to PeerConnection, which makes more sense for this functionality. An API parameter to set an upper limit to the size of the logfile is introduced.
> The old interface on PeerConnectionFactory is not removed in this CL, because it is called from Chrome, it will be removed after Chrome is updated to use the PeerConnection interface.
>
> BUG=webrtc:4741,webrtc:5603,chromium:609749
> R=solenberg@webrtc.org, stefan@webrtc.org, terelius@webrtc.org, tommi@webrtc.org
>
> Committed: https://crrev.com/1895526c6130e3d0e9b154f95079b8eda7567016
> Cr-Commit-Position: refs/heads/master@{#13321}

TBR=solenberg@webrtc.org,tommi@webrtc.org,stefan@webrtc.org,terelius@webrtc.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG=webrtc:4741,webrtc:5603,chromium:609749

Review-Url: https://codereview.webrtc.org/2111813002
Cr-Commit-Position: refs/heads/master@{#13340}
2016-06-30 07:59:49 +00:00
ca4cac7e74 New module for the adaptive level controlling functionality in the audio processing module
NOTRY=true
TBR=aluebs@webrtc.org
BUG=webrtc:5920

Review-Url: https://codereview.webrtc.org/2090583002
Cr-Commit-Position: refs/heads/master@{#13333}
2016-06-29 22:26:19 +00:00
37f93af032 Remove unused constructor from VCMPacket.
BUG=
R=pbos@webrtc.org, sprang@webrtc.org

Review URL: https://codereview.webrtc.org/2110663004 .

Cr-Commit-Position: refs/heads/master@{#13325}
2016-06-29 16:03:20 +00:00
1895526c61 Move RtcEventLog object from inside VoiceEngine to Call.
In addition to moving the logging object itself, this also moves the interface from PeerConnectionFactory to PeerConnection, which makes more sense for this functionality. An API parameter to set an upper limit to the size of the logfile is introduced.
The old interface on PeerConnectionFactory is not removed in this CL, because it is called from Chrome, it will be removed after Chrome is updated to use the PeerConnection interface.

BUG=webrtc:4741,webrtc:5603,chromium:609749
R=solenberg@webrtc.org, stefan@webrtc.org, terelius@webrtc.org, tommi@webrtc.org

Review URL: https://codereview.webrtc.org/1748403002 .

Cr-Commit-Position: refs/heads/master@{#13321}
2016-06-29 11:57:01 +00:00
a3c51ea9f4 Revert "Pull out the PostFilter to its own NonlinearBeamformer API"
This reverts commit b983112bc7686ed4276def4c9215c498444c6bdd.

It was breaking dependencies.
TBR=aluebs@webrtc.org
BUG=

Review URL: https://codereview.webrtc.org/2110503002 .

Cr-Commit-Position: refs/heads/master@{#13316}
2016-06-28 17:38:41 +00:00
b983112bc7 Pull out the PostFilter to its own NonlinearBeamformer API
This is done to avoid having a nonlinear component in the AEC path.
Now the linear delay and sum is run before the AEC and the postfilter after it.

R=henrik.lundin@webrtc.org, peah@webrtc.org

Review URL: https://codereview.webrtc.org/1982183002 .

Cr-Commit-Position: refs/heads/master@{#13314}
2016-06-28 17:02:57 +00:00
31b2ec4e0d Remove unused output parameter in VCMReceiver::FrameForDecoding().
BUG=
R=pbos@webrtc.org

Review URL: https://codereview.webrtc.org/2104863002 .

Cr-Commit-Position: refs/heads/master@{#13310}
2016-06-28 11:32:57 +00:00
95348f7663 Remove unused parameters from VCMReceiver::InsertPacket().
BUG=
R=pbos@webrtc.org

Review URL: https://codereview.webrtc.org/2094183004 .

Cr-Commit-Position: refs/heads/master@{#13309}
2016-06-28 09:11:37 +00:00
098e6c5d0a Logging and tracing of audio devices on Andriod.
Replaced invokations of WEBRTC_TRACE with LOG, which is
visible in the android system log.

BUG=NONE
R=henrika@webrtc.org

Review URL: https://codereview.webrtc.org/2091803002 .

Cr-Commit-Position: refs/heads/master@{#13308}
2016-06-28 07:36:39 +00:00
3d2e853f86 Replace OS_ANDROID define to disable webrtc test.
Replaces OS_ANDROID with WEBRTC_ANDROID to disable
ProcessNoLossChangeFrameRateFrameDropVP9 on Android.

BUG=webrtc:6057
R=danilchap@webrtc.org
TBR=phoglund@webrtc.org

Review URL: https://codereview.webrtc.org/2101673002 .

Cr-Commit-Position: refs/heads/master@{#13300}
2016-06-27 17:47:20 +00:00
3532ee2f0a Disable flaky ProcessNoLossChangeFrameRateFrameDropVP9 on Android
BUG=webrtc:6057
R=pbos@webrtc.org

Review URL: https://codereview.webrtc.org/2100863002 .

Cr-Commit-Position: refs/heads/master@{#13298}
2016-06-27 16:17:21 +00:00