ef048b0d65
for issue #4418, #4151, #4076 .DVR Missing First Few Seconds of Audio/Video ### Root Cause When recording WebRTC streams to FLV files using DVR, the first 4-6 seconds of audio/video are missing. This occurs because: 1. **Packets are discarded before A/V sync is available**: The RTC-to-RTMP conversion pipeline actively discards all RTP packets when avsync_time <= 0. 2. **Original algorithm requires 2 RTCP SR packets**: The previous implementation needed to receive two RTCP Sender Report (SR) packets before it could calculate the rate for audio/video synchronization timestamp conversion. 3. **Delay causes packet loss**: Since RTCP SR packets typically arrive every 2-3 seconds, waiting for 2 SRs means 4-6 seconds of packets are discarded before A/V sync becomes available. 4. **Audio SR arrives slower than video SR**: As reported in the issue, video RTCP SR packets arrive much faster than audio SR packets. This asymmetry causes audio packets to be discarded for a longer period, resulting in the audio loss observed in DVR recordings. ### Solution 1. **Initialize rate from SDP**: Use the sample rate from SDP (Session Description Protocol) to calculate the initial rate immediately when the track is created. Audio (Opus): 48000 Hz → rate = 48 (RTP units per millisecond) Video (H.264/H.265): 90000 Hz → rate = 90 (RTP units per millisecond) 2. **Enable immediate A/V sync:** With the SDP rate available, cal_avsync_time() can calculate valid timestamps from the very first RTP packet, eliminating packet loss. 3. **Smooth transition to precise rate**: After receiving the 2nd RTCP SR, update to the precisely calculated rate based on actual RTP/NTP timestamp mapping. ## Configuration Added new configuration option `init_rate_from_sdp` in the RTC vhost section: ```nginx vhost rtc.vhost.srs.com { rtc { # Whether initialize RTP rate from SDP sample rate for immediate A/V sync. # When enabled, the RTP rate (units per millisecond) is initialized from the SDP # sample rate (e.g., 90 for video 90kHz, 48 for audio 48kHz) before receiving # 2 RTCP SR packets. This allows immediate audio/video synchronization. # The rate will be updated to a more precise value after receiving the 2nd SR. # Overwrite by env SRS_VHOST_RTC_INIT_RATE_FROM_SDP for all vhosts. # Default: off init_rate_from_sdp off; } } ``` **⚠️ Important Note**: This config defaults to **off** because: - ✅ When **enabled**: Fixes the audio loss problem (no missing first 4-6 seconds) - ❌ When **enabled**: VLC on macOS cannot play the video properly - ✅ Other platforms work fine (Windows, Linux) - ✅ FFplay works fine on all platforms Users experiencing audio loss in DVR recordings can enable this option if they don't need VLC macOS compatibility. We're investigating the VLC macOS issue to make this feature safe to enable by default in the future. --------- Co-authored-by: winlin <winlinvip@gmail.com> Co-authored-by: OSSRS-AI <winlinam@gmail.com>
84 lines
2.3 KiB
C++
84 lines
2.3 KiB
C++
//
|
|
// Copyright (c) 2013-2025 The SRS Authors
|
|
//
|
|
// SPDX-License-Identifier: MIT
|
|
//
|
|
|
|
#ifndef SRS_UTEST_AI09_HPP
|
|
#define SRS_UTEST_AI09_HPP
|
|
|
|
/*
|
|
#include <srs_utest_ai09.hpp>
|
|
*/
|
|
#include <srs_utest.hpp>
|
|
|
|
#include <srs_app_rtc_codec.hpp>
|
|
#include <srs_app_rtc_source.hpp>
|
|
#include <srs_kernel_rtc_rtp.hpp>
|
|
#include <srs_protocol_amf0.hpp>
|
|
#include <srs_protocol_format.hpp>
|
|
#include <srs_protocol_rtmp_stack.hpp>
|
|
|
|
// Forward declarations
|
|
class SrsMediaPacket;
|
|
class SrsRtpPacket;
|
|
|
|
// Mock frame target for testing SrsRtcFrameBuilder
|
|
class MockRtcFrameTarget : public ISrsFrameTarget
|
|
{
|
|
public:
|
|
int on_frame_count_;
|
|
SrsMediaPacket *last_frame_;
|
|
srs_error_t frame_error_;
|
|
|
|
public:
|
|
MockRtcFrameTarget();
|
|
virtual ~MockRtcFrameTarget();
|
|
virtual srs_error_t on_frame(SrsMediaPacket *frame);
|
|
void reset();
|
|
};
|
|
|
|
// Mock audio transcoder for testing SrsRtcFrameBuilder::transcode_audio
|
|
class MockAudioTranscoderForUtest : public ISrsAudioTranscoder
|
|
{
|
|
public:
|
|
// Control behavior
|
|
srs_error_t transcode_error_;
|
|
std::vector<SrsParsedAudioPacket *> output_packets_;
|
|
bool should_output_packets_;
|
|
uint8_t *aac_header_data_;
|
|
int aac_header_len_;
|
|
|
|
public:
|
|
MockAudioTranscoderForUtest();
|
|
virtual ~MockAudioTranscoderForUtest();
|
|
|
|
public:
|
|
// ISrsAudioTranscoder interface
|
|
virtual srs_error_t initialize(SrsAudioCodecId from, SrsAudioCodecId to, int channels, int sample_rate, int bit_rate);
|
|
virtual srs_error_t transcode(SrsParsedAudioPacket *in, std::vector<SrsParsedAudioPacket *> &outs);
|
|
virtual void free_frames(std::vector<SrsParsedAudioPacket *> &frames);
|
|
virtual void aac_codec_header(uint8_t **data, int *len);
|
|
|
|
public:
|
|
// Test helpers
|
|
void reset();
|
|
void set_output_packets(int count, const char *sample_data = NULL, int sample_size = 64);
|
|
void set_transcode_error(srs_error_t err);
|
|
};
|
|
|
|
// Mock request class for testing - implement ISrsRequest interface
|
|
class MockRtcRequest : public ISrsRequest
|
|
{
|
|
public:
|
|
MockRtcRequest(std::string vhost = "__defaultVhost__", std::string app = "live", std::string stream = "test");
|
|
virtual ~MockRtcRequest();
|
|
virtual ISrsRequest *copy();
|
|
virtual std::string get_stream_url();
|
|
virtual void update_auth(ISrsRequest *req);
|
|
virtual void strip();
|
|
virtual ISrsRequest *as_http();
|
|
};
|
|
|
|
#endif
|