zqz/platform-external-webrtc
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Skip frame with unknown frame id in receiver part of DefaultVideoQualityAnalyzer

Browse Source 
It may happen that if we have simulcast track with, let's say, 2 streams
A and B, we can receive frame X on A and then receive it again on B
when there is a switch from A to B. TO correctly handle it we need to
skip second receive of X. Later we need to add metric which will show
how many frames were in between when X was received twice.

Bug: webrtc:11557
Change-Id: I8c52a78674b62387f520a587f51e209ed7c0b0bc
Reviewed-on: https://webrtc-review.googlesource.com/c/src/+/176853
Commit-Queue: Artem Titov <titovartem@webrtc.org>
Reviewed-by: Ilya Nikolaevskiy <ilnik@webrtc.org>
Cr-Commit-Position: refs/heads/master@{#31488}
This commit is contained in:
Artem Titov
2020-06-10 14:09:53 +02:00
committed by Commit Bot
parent 71db9acc40
commit 5f7bfbe6c6
2 changed files with 30 additions and 6 deletions
Download Patch File Download Diff File Expand all files Collapse all files

34
test/pc/e2e/analyzer/video/default_video_quality_analyzer.cc
View File

@ -219,7 +219,14 @@ void DefaultVideoQualityAnalyzer::OnFramePreDecode(
const webrtc::EncodedImage& input_image) {
rtc::CritScope crit(&lock_);
auto it = frame_stats_.find(frame_id);
RTC_DCHECK(it != frame_stats_.end());
if (it == frame_stats_.end()) {
// It means this frame was predecoded before, so we can skip it. It may
// happen when we have multiple simulcast streams in one track and received
// the same picture from two different streams because SFU can't reliably
// correlate two simulcast streams and started relaying the second stream
// from the same frame it has relayed right before for the first stream.
return;
}
RTC_DCHECK(it->second.received_time.IsInfinite())
<< "Received multiple spatial layers for stream_label="
<< it->second.stream_label;
@ -244,7 +251,14 @@ void DefaultVideoQualityAnalyzer::OnFrameDecoded(
const DecoderStats& stats) {
rtc::CritScope crit(&lock_);
auto it = frame_stats_.find(frame.id());
RTC_DCHECK(it != frame_stats_.end());
if (it == frame_stats_.end()) {
// It means this frame was decoded before, so we can skip it. It may happen
// when we have multiple simulcast streams in one track and received
// the same picture from two different streams because SFU can't reliably
// correlate two simulcast streams and started relaying the second stream
// from the same frame it has relayed right before for the first stream.
return;
}
frame_counters_.decoded++;
stream_frame_counters_[it->second.stream_label].decoded++;
it->second.decode_end_time = Now();
@ -253,16 +267,24 @@ void DefaultVideoQualityAnalyzer::OnFrameDecoded(
void DefaultVideoQualityAnalyzer::OnFrameRendered(
absl::string_view peer_name,
const webrtc::VideoFrame& raw_frame) {
rtc::CritScope crit(&lock_);
auto stats_it = frame_stats_.find(raw_frame.id());
if (stats_it == frame_stats_.end()) {
// It means this frame was rendered before, so we can skip it. It may happen
// when we have multiple simulcast streams in one track and received
// the same picture from two different streams because SFU can't reliably
// correlate two simulcast streams and started relaying the second stream
// from the same frame it has relayed right before for the first stream.
return;
}
FrameStats* frame_stats = &stats_it->second;
// Copy entire video frame including video buffer to ensure that analyzer
// won't hold any WebRTC internal buffers.
VideoFrame frame = raw_frame;
frame.set_video_frame_buffer(
I420Buffer::Copy(*raw_frame.video_frame_buffer()->ToI420()));
rtc::CritScope crit(&lock_);
auto stats_it = frame_stats_.find(frame.id());
RTC_DCHECK(stats_it != frame_stats_.end());
FrameStats* frame_stats = &stats_it->second;
// Update frames counters.
frame_counters_.rendered++;
stream_frame_counters_[frame_stats->stream_label].rendered++;

2
test/pc/e2e/analyzer/video/single_process_encoded_image_data_injector.cc
View File

@ -95,6 +95,8 @@ EncodedImageExtractionResult SingleProcessEncodedImageDataInjector::ExtractData(
{
rtc::CritScope crit(&lock_);
auto ext_vector_it = extraction_cache_.find(next_id);
// TODO(titovartem) add support for receiving single frame multiple times
// when in simulcast key frame for another spatial stream can be received.
RTC_CHECK(ext_vector_it != extraction_cache_.end())
<< "Unknown frame_id=" << next_id;