Commit Graph

270 Commits

Author SHA1 Message Date
Haibo Chen(陈海博)
ef048b0d65
RTC: Fix DVR missing first 4-6 seconds by initializing rate from SDP (#4541)
for issue #4418, #4151, #4076 .DVR Missing First Few Seconds of
Audio/Video

### Root Cause
When recording WebRTC streams to FLV files using DVR, the first 4-6
seconds of audio/video are missing. This occurs because:

1. **Packets are discarded before A/V sync is available**: The
RTC-to-RTMP conversion pipeline actively discards all RTP packets when
avsync_time <= 0.
2. **Original algorithm requires 2 RTCP SR packets**: The previous
implementation needed to receive two RTCP Sender Report (SR) packets
before it could calculate the rate for audio/video synchronization
timestamp conversion.
3. **Delay causes packet loss**: Since RTCP SR packets typically arrive
every 2-3 seconds, waiting for 2 SRs means 4-6 seconds of packets are
discarded before A/V sync becomes available.
4. **Audio SR arrives slower than video SR**: As reported in the issue,
video RTCP SR packets arrive much faster than audio SR packets. This
asymmetry causes audio packets to be discarded for a longer period,
resulting in the audio loss observed in DVR recordings.

### Solution
1. **Initialize rate from SDP**: Use the sample rate from SDP (Session
Description Protocol) to calculate the initial rate immediately when the
track is created.
Audio (Opus): 48000 Hz → rate = 48 (RTP units per millisecond)
Video (H.264/H.265): 90000 Hz → rate = 90 (RTP units per millisecond)
2. **Enable immediate A/V sync:** With the SDP rate available,
cal_avsync_time() can calculate valid timestamps from the very first RTP
packet, eliminating packet loss.
3. **Smooth transition to precise rate**: After receiving the 2nd RTCP
SR, update to the precisely calculated rate based on actual RTP/NTP
timestamp mapping.

## Configuration

Added new configuration option `init_rate_from_sdp` in the RTC vhost
section:

```nginx
vhost rtc.vhost.srs.com {
    rtc {
        # Whether initialize RTP rate from SDP sample rate for immediate A/V sync.
        # When enabled, the RTP rate (units per millisecond) is initialized from the SDP
        # sample rate (e.g., 90 for video 90kHz, 48 for audio 48kHz) before receiving
        # 2 RTCP SR packets. This allows immediate audio/video synchronization.
        # The rate will be updated to a more precise value after receiving the 2nd SR.
        # Overwrite by env SRS_VHOST_RTC_INIT_RATE_FROM_SDP for all vhosts.
        # Default: off
        init_rate_from_sdp off;
    }
}
```

**⚠️ Important Note**: This config defaults to **off** because:
-  When **enabled**: Fixes the audio loss problem (no missing first 4-6
seconds)
-  When **enabled**: VLC on macOS cannot play the video properly
-  Other platforms work fine (Windows, Linux)
-  FFplay works fine on all platforms

Users experiencing audio loss in DVR recordings can enable this option
if they don't need VLC macOS compatibility. We're investigating the VLC
macOS issue to make this feature safe to enable by default in the
future.

---------

Co-authored-by: winlin <winlinvip@gmail.com>
Co-authored-by: OSSRS-AI <winlinam@gmail.com>
2025-10-28 09:33:40 -04:00
OSSRS-AI
054d3a3563 AI: Add workflow utest for rtc conn. 2025-10-17 21:55:29 -04:00
OSSRS-AI
702a58df6a AI: Improve coverage for app rtmp module. 2025-10-03 10:10:57 -04:00
OSSRS-AI
c0fc8cb093 AI: Improve converage for app rtc module. 2025-09-27 09:40:57 -04:00
Haibo Chen(陈海博)
ea14caeee5
rename HEVC-related mux functions to enhance consistency and readability. (#4506) 2025-09-22 07:48:40 -04:00
winlin
10c0b66c0f Fix WHIP with transcoding bug. v7.0.92 (#4495) 2025-09-21 08:58:22 -04:00
Winlin
20f6cd595c
AI: Refine RTMP/SRT/RTC bridge. v7.0.90 (#4503)
This PR refactors the stream bridge architecture in SRS to improve code
organization, type safety, and maintainability by replacing the generic
ISrsStreamBridge interface with protocol-specific bridge classes and
target interfaces.

1. New Target Interface Architecture:

- Introduces  ISrsFrameTarget for AV frame consumers (RTMP sources)
- Introduces  ISrsRtpTarget for RTP packet consumers (RTC sources)
- Introduces ISrsSrtTarget for SRT packet consumers (SRT sources)

2. Protocol-Specific Bridge Classes:

- SrsRtmpBridge: Converts RTMP frames to RTC/RTSP protocols
-  SrsSrtBridge: Converts SRT packets to RTMP/RTC protocols
-  SrsRtcBridge: Converts RTC packets to RTMP protocol

3. Simplified Bridge Management:

- Removes the generic SrsCompositeBridge chain pattern
- Each source type now uses its appropriate bridge type directly

With this improvement, you are able to implement very complex bridge and
protocol converting, for example, you can bridge RTMP to RTC with opus
audio when you support enhanced RTMP with opus.

Another plan is to support bridging RTC to RTSP, directly without
converting RTP to media frame packet, but directly deliver RTP packet
from RTC source to RTSP source.

---------

Co-authored-by: OSSRS-AI <winlinam@gmail.com>
2025-09-19 21:50:28 -04:00
Winlin
e999de09ea AI: Add utests to cover app rtc module. (#4498)
Co-authored-by: OSSRS-AI <winlinam@gmail.com>
2025-09-18 11:30:28 -04:00
Winlin
04b88e889f AI: Improve coverage of app by utest (#4494)
Co-authored-by: OSSRS-AI <winlinam@gmail.com>
2025-09-17 21:51:07 -04:00
Winlin
5b27c3fa7a RTC2RTMP: Fix sequence number wraparound assertion crashes. v6.0.177 v7.0.89 (#4491)
The issue occurred when srs_rtp_seq_distance(start, end) + 1 resulted in
values <= 0
due to sequence number wraparound (e.g., when end < start). This caused
assertion
failures and server crashes.

SrsRtcFrameBuilder::check_frame_complete(): Added validation to return
false
  for invalid sequence ranges instead of asserting.

However, it maybe cause converting RTC to RTMP stream failure, because
this issue
should be caused by the problem of sequence number of RTP, which means
there potentially
be stream problem in RTC stream. Even so, changing assert to warning
logs is better,
because SRS should not crash when stream is corrupt.

---------

Co-authored-by: OSSRS-AI <winlinam@gmail.com>
2025-09-15 11:02:30 -04:00
Winlin
d4d1d5d8b5
AI: Move some app files to kernel. v7.0.86 (#4486)
Co-authored-by: OSSRS-AI <winlinam@gmail.com>
2025-09-13 10:26:47 -04:00
Winlin
2384f3fb06
AI: Fix naming problem for app module. v7.0.85 (#4485)
Co-authored-by: OSSRS-AI <winlinam@gmail.com>
2025-09-12 19:44:43 -04:00
Winlin
3a29e5c550
AI: Fix naming issue for protocol module. v7.0.83 (#4482)
Co-authored-by: OSSRS-AI <winlinam@gmail.com>
2025-09-09 21:06:45 -04:00
Winlin
8f87d4092b
AI: Fix naming problem in kernel module. v7.0.82 (#4479)
Co-authored-by: OSSRS-AI <winlinam@gmail.com>
2025-09-07 21:09:08 -04:00
Winlin
8976ce4c8d
AI: Support anonymous coroutine with code block. v7.0.80 (#4475)
This PR introduces anonymous coroutine macros for easier coroutine
creation and improves the State Threads (ST) mutex and condition
variable handling in SRS.

- **Added coroutine macros**: `SRS_COROUTINE_GO`, `SRS_COROUTINE_GO2`,
`SRS_COROUTINE_GO_CTX`, `SRS_COROUTINE_GO_CTX2`
- **Added `SrsCoroutineChan`**: Channel for sharing data between
coroutines with coroutine-safe operations
- **Simplified coroutine creation**: Go-like syntax for creating
anonymous coroutines with code blocks

---------

Co-authored-by: Jacob Su <suzp1984@gmail.com>
Co-authored-by: OSSRS-AI <winlinam@gmail.com>
2025-09-06 08:10:49 -04:00
why
57e1622e81
WebRTC: Fix NACK recovered packets not being added to receive queue. v7.0.78 (#4467)
Fixes a bug in WebRTC NACK packet recovery mechanism where recovered
packets were being discarded instead of processed.

In `SrsRtcRecvTrack::on_nack()`, when a retransmitted packet arrived
(found in NACK receiver), the method would:
1.  Remove the packet from NACK receiver (correct)
2.  Return early without adding the packet to RTP queue (BUG)

This caused recovered packets to be lost, defeating the purpose of the
NACK mechanism and potentially causing media quality issues.

Restructured the control flow in `on_nack()` to ensure both new and
recovered packets reach the packet insertion logic:

- **Before**: Early return for recovered packets → packets discarded
- **After**: Conditional NACK management + unified packet processing →
all packets queued

Closes #3820

---------

Co-authored-by: Haibo Chen <495810242@qq.com>
Co-authored-by: OSSRS-AI <winlinam@gmail.com>
2025-09-04 08:07:36 -04:00
winlin
f3059d37a4 Refine RTMP common message. 2025-09-01 18:51:20 -04:00
Winlin
3e8cb3f9d5
AI: Replace SrsSharedPtrMessage with SrsMediaPacket for unified media packet handling. v7.0.74 (#4465)
This PR introduces a major refactoring to replace `SrsSharedPtrMessage`
with `SrsMediaPacket` throughout the SRS codebase, providing a more
unified and cleaner approach to media packet handling.

---------

Co-authored-by: OSSRS-AI <winlinam@gmail.com>
2025-09-01 18:06:24 -04:00
Winlin
c534a265e5
AI: Update RTMP message memory management with shared pointers. v7.0.73 (#4464)
This PR modernizes the memory management architecture in SRS by
refactoring RTMP message handling to use shared pointers
(SrsSharedPtr<SrsMemoryBlock>) instead of manual memory management. This
change improves memory safety, reduces the risk of memory leaks, and
provides a cleaner abstraction for message payload handling.

* Introduced `SrsMemoryBlock`: A dedicated class for managing memory
buffers with size information
* Replaced manual memory management: `SrsCommonMessage` and
`SrsSharedPtrMessage` now use `SrsSharedPtr<SrsMemoryBlock>` instead of
raw pointers
* Updated `SrsRtpPacket`: Now uses `SrsSharedPtr<SrsMemoryBlock>` for
shared buffer management

---------

Co-authored-by: OSSRS-AI <winlinam@gmail.com>
2025-09-01 14:00:31 -04:00
Winlin
728828e1dd
AI: Extract shared components and improve SRS server architecture. v7.0.70 (#4461)
Move global xpps statistics variables from `srs_app_server.cpp` to
`srs_kernel_kbps.cpp`.

Extract global shared timers from `SrsServer` into new `SrsSharedTimer`
class.

Extract WebRTC session management logic from `SrsServer` into dedicated
`SrsRtcSessionManager` class.

Extract PID file handling into dedicated  `SrsPidFileLocker` class.

---------

Co-authored-by: OSSRS-AI <winlinam@gmail.com>
2025-08-31 19:14:34 -04:00
Winlin
32dfed43ef
AI: Merge SRT and RTC servers into unified SrsServer. v7.0.68 (#4459)
This PR consolidates the SRT and RTC server functionality into the main
SrsServer class, eliminating the separate `SrsSrtServer` and
`SrsRtcServer` classes and their corresponding adapter classes. This
architectural change simplifies the codebase by removing the hybrid
server pattern and integrating all protocol handling directly into
`SrsServer`.

As unified connection manager (`_srs_conn_manager`) for all protocol
connections, all incoming connections are checked against the same
connection limit in `on_before_connection()`. This enables consistent
connection limits: `max_connections` now protects against resource
exhaustion from any protocol, not just RTMP.

Remove modules because it's not used now, so only keep the server
application module and main entry point. Remove the wait group to run
server, instead, directly run server and invoke the cycle method.

After this PR, the startup workflow and servers architecture should be
much easier to maintain.

---------

Co-authored-by: OSSRS-AI <winlinam@gmail.com>
2025-08-31 08:58:37 -04:00
Winlin
1fa2cba7c0
Organize utility functions to kernel. v7.0.65 (#4455) 2025-08-27 21:35:58 -04:00
Winlin
72ddc28d97
AI: Implement stream publish token system to prevent race conditions across all protocols. v7.0.62 (#4452)
This PR introduces a comprehensive stream publish token system that
prevents race conditions when multiple publishers attempt to publish to
the same stream URL simultaneously across different protocols (RTMP,
WebRTC, SRT).

* Race Condition Issue: Multiple publishers could create duplicate
sources for the same stream when context switches occurred during source
initialization in SRS's coroutine-based architecture
* Cross-Protocol Conflicts: Different protocols (RTMP, RTC, SRT) could
simultaneously publish to the same stream URL without coordination
* Resource Management: No centralized mechanism to ensure exclusive
stream publishing access

---------

Co-authored-by: OSSRS-AI <winlinam@gmail.com>
2025-08-26 10:27:53 -04:00
Winlin
1b6f97bd2d
Refine source lock to fix race condition in source managers. v7.0.61 (#4449)
This PR fixes a critical race condition in SRS source managers where
multiple coroutines could create duplicate sources for the same stream.

- **Atomic source creation**: Source lookup, creation, and pool
insertion now happen atomically within lock scope
- **Consistent interface**: Standardize on `ISrsRequest*` interface
throughout codebase
- **Handler simplification**: Remove `ISrsLiveSourceHandler*` parameter,
obtain from global server instance

---------

Co-authored-by: OSSRS-AI <winlinam@gmail.com>
2025-08-23 07:36:41 -06:00
Winlin
6ec97067de
AI: Remove cygwin64, always enable WebRTC, and enforce C++98 compatibility. v7.0.60 (#4447)
This PR makes WebRTC a core feature of SRS and enforces C++98
compatibility by:

1. Always Enable WebRTC Support
- Remove `--rtc=on|off` configuration option - WebRTC is now always
enabled
- Eliminate all `#ifdef SRS_RTC` conditional compilation blocks
- Include WebRTC-related modules (RTC, SRTP, DTLS) in all builds
- Update build scripts to always link WebRTC dependencies

2. Enforce C++98 Compatibility  
- Remove `--cxx11=on|off` and `--cxx14=on|off` configuration options
- Force `SRS_CXX11=NO` and `SRS_CXX14=NO` in build system
- Move these options to deprecated section with warnings
- Ensure codebase maintains C++98 standard compatibility

3. Remove Windows/Cygwin Support
- Remove all Windows and Cygwin64 conditional compilation blocks (#ifdef
_WIN32, #ifdef CYGWIN64)
- Delete Cygwin64 build configurations from build scripts (
auto/options.sh, auto/depends.sh, configure)
- Remove Cygwin64 assembly files and State Threads platform support (
md_cygwin64.S)
- Eliminate Windows-specific GitHub Actions workflows and CI/CD jobs
- Remove NSIS packaging files and Windows installer generation
- Delete Windows documentation and update feature lists to mark support
as removed in v7.0
- Simplify OS detection to only support Unix-like systems (Linux, macOS)

4. Code Cleanup
- Remove conditional WebRTC code blocks throughout the codebase
- Simplify build configuration by removing WebRTC-related conditionals
- Update constructor delegation patterns to be C++98 compatible
- Fix vector initialization to use C++98 syntax
- Eliminate Windows-specific implementations for file operations, time
handling, and networking
- Unified platform handling with consistent POSIX API usage

---------

Co-authored-by: OSSRS-AI <winlinam@gmail.com>
2025-08-21 10:03:38 -06:00
Winlin
5adf684f59
AI: Remove multi-threading support and change to single-thread architecture. v7.0.59 (#4445)
This PR removes the multi-threading infrastructure from SRS and
consolidates the codebase to use single-thread architecture exclusively.
This is a architectural simplification that aligns with SRS's
coroutine-based design philosophy.

* Simplified Architecture: Eliminates complexity of multi-threading
coordination
* Better Alignment: Matches SRS's coroutine-based single-thread design
philosophy
* Reduced Complexity: Removes potential race conditions and threading
bugs
* Cleaner Code: More focused modules with clear responsibilities
* Easier Maintenance: Fewer moving parts and clearer execution flow

---------

Co-authored-by: OSSRS-AI <winlinam@gmail.com>
2025-08-20 15:12:51 -06:00
Jacob Su
8cd4616147
fix err memory leak in rtc to rtmp bridge. v6.0.174 v7.0.55 (#4441)
1. print the error messages before dismiss it;
2. free the err to avoid memory leak;

## Cause
found this issue when research #4434 .


## Impact

 1. develop
 2. 5.0release
 3. 6.0release

---------

Co-authored-by: winlin <winlinvip@gmail.com>
2025-08-16 07:56:49 -06:00
Winlin
6e1134fe9b
Use clang format. v7.0.52 (#4433)
---------

Co-authored-by: ChenGH <chengh_math@126.com>
2025-08-11 23:19:19 -04:00
chundonglinlin
e712b12a15
RTC: audio packet jitter buffer. v7.0.48 (#4295)
Rtp packets may be retransmitted, disordered, jittery, delayed,
etc.There may be abnormalities when converting to rtmp.

To reproduce this problem, you need to set the network reordering by
[tc-ui](https://github.com/ossrs/tc-ui). Note that you need a linux
server, and start it by docker:

```bash
docker run --network=host --privileged -it --restart always -d \
    --name tc -v /lib/modules:/lib/modules:ro ossrs/tc-ui:1
```

Set up 5% packet reordering and a 1ms delay; then you will notice that
the audio is stuttering, somewhat noisy, and lacks fluency.

```bash
curl http://localhost:2023/tc/api/v1/config/raw -X POST \
  -d 'tcset ens5 --direction incoming --delay 40ms --reordering 5% --port 8000'
```

> Note: Even without network conditions, the natural state can also
cause packet reordering, especially in public cloud platforms such as
AWS EC2.

> Note: You can use command `curl
http://localhost:2023/tc/api/v1/config/raw -X POST -d 'tcdel --all
ens5'` to reset the network condition settings.

Check the web console, you will see the reordering setup:

<img width="500" alt="TC Settings"
src="https://github.com/user-attachments/assets/b278fdf4-9fcc-4aac-b534-dfa34e28c371"
/>

Then, publish stream via WHIP: http://localhost:8080/players/whip.html

And, play via HTTP-FLV: http://localhost:8080/players/srs_player.html

Finished by AI:

* [AI: Extract audio jitter buffer to class
AudioPacketCache](a4097d9374)
* [AI: Add utest and fix
bug.](c919227af5)

---------

Co-authored-by: Haibo Chen <495810242@qq.com>
Co-authored-by: winlin <winlinvip@gmail.com>
2025-07-16 21:36:56 -04:00
Haibo Chen(陈海博)
5dc292ce64
NEW PROTOCOL: Support viewing stream over RTSP. v7.0.47 (#4333)
## Introduce

This PR adds support for viewing streams via the RTSP protocol. Note
that it only supports viewing streams, not publishing streams via RTSP.

Currently, only publishing via RTMP is supported, which is then
converted to RTSP. Further work is needed to support publishing RTC/SRT
streams and converting them to RTSP.

## Usage

Build and run SRS with RTSP support:

```
cd srs/trunk && ./configure --rtsp=on && make -j16
./objs/srs -c conf/rtsp.conf
```

Push stream via RTMP by FFmpeg:

```
ffmpeg -re -i doc/source.flv -c copy -f flv rtmp://localhost/live/livestream
```

View the stream via RTSP protocol, try UDP first, then use TCP:

```
ffplay -i rtsp://localhost:8554/live/livestream
```

Or specify the transport protocol with TCP:

```
ffplay -rtsp_transport tcp -i rtsp://localhost:8554/live/livestream
```

## Unit Test

Run utest for RTSP:

```
./configure --utest=on & make utest -j16
./objs/srs_utest
```

## Regression Test

You need to start SRS for regression testing.

```
./objs/srs -c conf/regression-test-for-clion.conf
```

Then run regression tests for RTSP.

```
cd srs/trunk/3rdparty/srs-bench
go test ./srs -mod=vendor -v -count=1 -run=TestRtmpPublish_RtspPlay
```

## Blackbox Test

For blackbox testing, SRS will be started by utest, so there is no need
to start SRS manually.

```
cd srs/trunk/3rdparty/srs-bench
go test ./blackbox -mod=vendor -v -count=1 -run=TestFast_RtmpPublish_RtspPlay_Basic
```

## UDP Transport

As UDP requires port allocation, this PR doesn't support delivering
media stream via UDP transport, so it will fail if you try to use UDP as
transport:

```
ffplay -rtsp_transport udp -i rtsp://localhost:8554/live/livestream

[rtsp @ 0x7fbc99a14880] method SETUP failed: 461 Unsupported Transport
rtsp://localhost:8554/live/livestream: Protocol not supported

[2025-07-05 21:30:52.738][WARN][14916][7d7gf623][35] RTSP: setup failed: code=2057
(RtspTransportNotSupported) : UDP transport not supported, only TCP/interleaved mode is supported
```

There are no plans to support UDP transport for RTSP. In the real world,
UDP is rarely used; the vast majority of RTSP traffic uses TCP.

## Play Before Publish

RTSP supports audio with AAC and OPUS codecs, which is significantly
different from RTMP or WebRTC.

RTSP uses commands to exchange SDP and specify the audio track to play,
unlike WHEP or HTTP-FLV, which use the query string of the URL. RTSP
depends on the player’s behavior, making it very difficult to use and
describe.

Considering the feature that allows playing the stream before publishing
it, it requires generating some default parameters in the SDP. For OPUS,
the sample rate is 48 kHz with 2 channels, while AAC is more complex,
especially regarding the sample rate, which may be 44.1 kHz, 32 kHz, or
48 kHz.

Therefore, for RTSP, we cannot support play-then-publish. Instead, there
must already be a stream when playing it, so that the audio codec is
determined.

## Opus Codec

No Opus codec support for RTSP, because for RTC2RTSP, it always converts
RTC to RTMP frames, then converts them to RTSP packets. Therefore, the
audio codec is always AAC after converting RTC to RTMP.

This means the bridge architecture needs some changes. We need a new
bridge that binds to the target protocol. For example, RTC2RTMP converts
the audio codec, but RTC2RTSP keeps the original audio codec.

Furthermore, the RTC2RTMP bridge should also support bypassing the Opus
codec if we use enhanced-RTMP, which supports the Opus audio codec. I
think it should be configurable to either transcode or bypass the audio
codec. However, this is not relevant to RTSP.

## AI Contributor

Below commits are contributed by AI:

* [AI: Remove support for media transport via
UDP.](755686229f)
* [AI: Add crutial logs for each RTSP
stage.](9c8cbe7bde)
* [AI: Support AAC doec for
RTSP.](7d7cc12bae)
* [AI: Add option --rtsp for
RTSP.](f67414d9ee)
* [AI: Extract SrsRtpVideoBuilder for RTC and
RTSP.](562e76b904)

---------

Co-authored-by: Jacob Su <suzp1984@gmail.com>
Co-authored-by: winlin <winlinvip@gmail.com>
2025-07-11 08:18:40 -04:00
Winlin
b2a827f8cf
Refine code and add tests for #4289. v7.0.45 (#4412)
Use AI to understand, add comments, add utests, refactor code for PR
#4289
2025-07-04 17:26:12 -04:00
Winlin
c5b6b72876
RTMP2RTC: Support dual video track for bridge. v7.0.44 (#4413)
This PR refactors the RTMP to RTC bridge to support multiple video
tracks by implementing lazy initialization of audio and video tracks.
Instead of pre-determining track parameters during bridge construction,
tracks are now initialized dynamically when the first packet of each
type is received, allowing proper codec detection and track
configuration for dual video track scenarios.

Failed to view WHEP with HEVC before publishing RTMP, because the
default codec is AVC and will not be updated until the stream is
published. This enables better handling of streams with multiple video
tracks in RTMP to WebRTC bridging scenarios. Now, you are able to:

1. View WHEP with HEVC: Play with WebRTC:
http://localhost:8080/players/whep.html?schema=http&&codec=hevc
2. Then publish by RTMP: `ffmpeg -stream_loop -1 -re -i doc/source.flv
-c:v libx265 -profile:v main -preset fast -b:v 2000k -maxrate 2000k
-bufsize 2000k -bf 0 -c:a aac -b:a 48k -ar 44100 -ac 2 -f flv
rtmp://localhost/live/livestream`

Or publish RTMP with HEVC, then view by WHEP.

Note that if the codecs do not match, the error log will display RTC:
`Drop for ssrc xxxxxx not found`. For example, this can occur if you
publish RTMP with HEVC while viewing the stream with AVC.
2025-07-04 14:23:13 -04:00
Haibo Chen(陈海博)
cbc98dc0d9
rtc2rtmp: Support RTC-to-RTMP remuxing with HEVC. v7.0.43 (#4349)
**Introduce**

This pull request builds upon the foundation laid in
https://github.com/ossrs/srs/pull/4289 . While the previous work solely
implemented unidirectional HEVC support from RTMP to RTC, this
submission further enhances it by introducing support for the RTC to
RTMP direction.

**Usage**

Launch SRS with `rtc2rtmp.conf`

```bash
./objs/srs -c conf/rtc2rtmp.conf
```

**Push with WebRTC**

Upgrade browser to Chrome(136+) or Safari(18+), then open [WHIP
encoder](http://localhost:8080/players/whip.html?schema=http&&codec=hevc),
push stream with URL that enables HEVC by query string `codec=hevc`:

```bash
http://localhost:1985/rtc/v1/whip/?app=live&stream=livestream&codec=hevc
```

This query string `codec=hevc` is used to select the video codec, and
generate lines in the answer SDP.

```
m=video 9 UDP/TLS/RTP/SAVPF 49 123
a=rtpmap:49 H265/90000
```

The encoder log also show the codec:

```
Audio: opus, 48000HZ, channels: 2, pt: 111
Video: H265, 90000HZ, pt: 49
```

**Play with RTMP**

Play HEVC stream via RTMP.

```bash
ffplay -i rtmp://localhost/live/livestream
```

You will see the codec in logs:

```
  Stream #0:0: Audio: aac (LC), 48000 Hz, stereo, fltp
  Stream #0:1: Video: hevc (Main), yuv420p(tv, bt709), 320x240, 30 fps, 30 tbr, 1k tbn
```

You can also use [WHEP
player](http://localhost:8080/players/whep.html?schema=http&&codec=hevc)
to play the stream.

Important refactor with AI:

* [AI: Refactor packet cache for RTC frame
builder.](b8ffa1630e)
* [AI: Refactor the packet copy and free for
SrsRtcFrameBuilder](f3487b45d7)
* [AI: Refactor the frame detector for
SrsRtcFrameBuilder](4ffc1526b9)
* [AI: Refactor the packet_video_rtmp for
SrsRtcFrameBuilder](81f6aef4ed)
* [AI: Add utests for
SrsCodecPayload.codec](61eb1c0bfc)
* [AI: Add utests for VideoPacketCache in
SrsRtcFrameBuilder.](fd25480dfa)
* [AI: Add utests for VideoFrameDetector in
SrsRtcFrameBuilder.](b4aa977bbd)
* [AI: Add regression test for RTC2RTMP with
HEVC.](5259a2aac3)

---------

Co-authored-by: Jacob Su <suzp1984@gmail.com>
Co-authored-by: winlin <winlinvip@gmail.com>
2025-07-03 08:24:42 -04:00
Winlin
07163df9a4
Refactor code for #4349 for better review. (#4405)
Also for augment AI to review it.
2025-06-27 10:52:00 -04:00
winlin
dcde554907 Debugging: Drop the specified N original SRTP packet for testing NACK. 2025-06-15 10:01:08 -04:00
Haibo Chen(陈海博)
0c88ddbcdf rtmp2rtc: Support RTMP-to-WebRTC conversion with HEVC. v7.0.33 (#4289)
```bash
C:\Program Files\Google\Chrome\Application>"C:\Program Files\Google\Chrome\Application\chrome.exe" --enable-features=WebRtcAllowH265Receive --force-fieldtrials=WebRTC-Video-H26xPacketBuffer/Enabled

open -a "Google Chrome" --args --enable-features=WebRtcAllowH265Receive --force-fieldtrials=WebRTC-Video-H26xPacketBuffer/Enabled
```

> Note: The latest Chrome browser (version 136) fully enables this by
default, so there's no need to launch it with any extra parameters.

```bash
./objs/srs -c conf/rtmp2rtc.conf
```

```bash
ffmpeg -stream_loop -1 -re -i input.mp4 -c:v libx265 -preset fast -b:v 2000k -maxrate 2000k -bufsize 4000k -bf 0 -c:a aac -b:a 128k -ar 44100 -ac 2 -f flv rtmp://localhost/live/livestream
```

```bash
http://localhost:1985/rtc/v1/whep/?app=live&stream=livestream
```

![image](https://github.com/user-attachments/assets/bdbf4c67-b7e2-4dc6-92a1-93e2c78e00fe)

sendrecv offer
```bash
--enable-features=WebRtcAllowH265Send,PlatformHEVCEncoderSupport,WebRtcAllowH265Receive --force-fieldtrials=WebRTC-Video-H26xPacketBuffer/Enabled
```

sendonly offer
```bash
--enable-features=WebRtcAllowH265Send,PlatformHEVCEncoderSupport
```

recvonly offer
```bash
--enable-features=WebRtcAllowH265Receive --force-fieldtrials=WebRTC-Video-H26xPacketBuffer/Enabled
```

* Browser Test for supporting H265

https://webrtc.github.io/samples/src/content/peerconnection/change-codecs/

![image](https://github.com/user-attachments/assets/174476df-a7aa-4951-9880-56328ec75065)

* How to test Safari: https://github.com/ossrs/srs/pull/3441
* Debug in Safari

![image](https://github.com/user-attachments/assets/6cf94fca-e3ed-46d2-a102-a472f1699b4e)

---------

Co-authored-by: chundonglinlin <chundonglinlin@163.com>
Co-authored-by: winlin <winlinvip@gmail.com>
Co-authored-by: john <hondaxiao@tencent.com>

---------

Co-authored-by: chundonglinlin <chundonglinlin@163.com>
Co-authored-by: john <hondaxiao@tencent.com>
2025-05-14 07:49:04 -04:00
Haibo Chen(陈海博)
e00937e387
Fix memory leaks from errors skipping resource release. v7.0.32 (#4308)
---------

Co-authored-by: winlin <winlinvip@gmail.com>
Co-authored-by: john <hondaxiao@tencent.com>

---------

Co-authored-by: john <hondaxiao@tencent.com>
2025-04-30 12:09:31 +08:00
Winlin
4e55bc83b7
Support custom deleter for SrsUniquePtr. (#4309)
SrsUniquePtr does not support array or object created by malloc, because
we only use delete to dispose the resource. You can use a custom
function to free the memory allocated by malloc or other allocators.
```cpp
      char* p = (char*)malloc(1024);
      SrsUniquePtr<char> ptr(p, your_free_chars);
```

This is used to replace the SrsAutoFreeH. For example:
```cpp
      addrinfo* r = NULL;
      SrsAutoFreeH(addrinfo, r, freeaddrinfo);
      getaddrinfo("127.0.0.1", NULL, &hints, &r);
```

Now, this can be replaced by:
```cpp
      addrinfo* r = NULL;
      getaddrinfo("127.0.0.1", NULL, &hints, &r);
      SrsUniquePtr<addrinfo> r2(r, freeaddrinfo);
```

Please aware that there is a slight difference between SrsAutoFreeH and
SrsUniquePtr. SrsAutoFreeH will track the address of pointer, while
SrsUniquePtr will not.
```cpp
      addrinfo* r = NULL;
      SrsAutoFreeH(addrinfo, r, freeaddrinfo); // r will be freed even r is changed later.
      SrsUniquePtr<addrinfo> ptr(r, freeaddrinfo); // crash because r is an invalid pointer.
```

---------

Co-authored-by: Haibo Chen <495810242@qq.com>
Co-authored-by: john <hondaxiao@tencent.com>
2025-04-26 00:01:34 -04:00
ChenGH
13597d1b7f
update copyright to 2025. v5.0.218 v6.0.159 v7.0.21 (#4271)
update copyright to 2025

---------

Co-authored-by: john <hondaxiao@tencent.com>
Co-authored-by: winlin <winlinvip@gmail.com>
2025-01-14 17:35:18 +08:00
Jacob Su
101382afd0
RTC2RTMP: Fix screen sharing stutter caused by packet loss. v5.0.216 v6.0.157 v7.0.18 (#4160)
## How to reproduce?

1. Refer this commit, which contains the web demo to capture screen as
video stream through RTC.
2. Copy the `trunk/research/players/whip.html` and
`trunk/research/players/js/srs.sdk.js` to replace the `develop` branch
source code.
3. `./configure && make`
4. `./objs/srs -c conf/rtc2rtmp.conf`
5. open `http://localhost:8080/players/whip.html?schema=http`
6. check `Screen` radio option.
7. click `publish`, then check the screen to share.
8. play the rtmp live stream: `rtmp://localhost/live/livestream`
9. check the video stuttering.

## Cause
When capture screen by the chrome web browser, which send RTP packet
with empty payload frequently, then all the cached RTP packets are
dropped before next key frame arrive in this case.

The OBS screen stream and camera stream do not have such problem.

## Add screen stream to WHIP demo

><img width="581" alt="Screenshot 2024-08-28 at 2 49 46 PM"
src="https://github.com/user-attachments/assets/9557dbd2-c799-4dfd-b336-5bbf2e4f8fb8">

---------

Co-authored-by: winlin <winlinvip@gmail.com>
2024-10-15 19:00:07 +08:00
Winlin
23d2602c34
UniquePtr: Support SrsUniquePtr to replace SrsAutoFree. v6.0.136 (#4109)
To manage an object:

```cpp
// Before
MyClass* ptr = new MyClass();
SrsAutoFree(MyClass, ptr);
ptr->do_something();

// Now
SrsUniquePtr<MyClass> ptr(new MyClass());
ptr->do_something();
```

To manage an array of objects:

```cpp
// Before
char* ptr = new char[10];
SrsAutoFreeA(char, ptr);
ptr[0] = 0xf;

// Now
SrsUniquePtr<char[]> ptr(new char[10]);
ptr[0] = 0xf;
```

In fact, SrsUniquePtr is a limited subset of SrsAutoFree, mainly
managing pointers and arrays. SrsUniquePtr is better than SrsAutoFree
because it has the same API to standard unique ptr.

```cpp
SrsUniquePtr<MyClass> ptr(new MyClass());
ptr->do_something();
MyClass* p = ptr.get();
```

SrsAutoFree actually uses a pointer to a pointer, so it can be set to
NULL, allowing the pointer's value to be changed later (this usage is
different from SrsUniquePtr).

```cpp
// OK to free ptr correctly.
MyClass* ptr;
SrsAutoFree(MyClass, ptr);
ptr = new MyClass();

// Crash because ptr is an invalid pointer.
MyClass* ptr;
SrsUniquePtr<MyClass> ptr(ptr);
ptr = new MyClass();
```

Additionally, SrsAutoFreeH can use specific release functions, which
SrsUniquePtr does not support.

---------

Co-authored-by: Jacob Su <suzp1984@gmail.com>
2024-07-09 10:29:36 +08:00
Jacob Su
75ddd8f5b6
Fix misspelling error in app config. v6.0.133 (#4077)
1. misspelling fix;
2. remove finished TODO;

---------

Co-authored-by: Haibo Chen <495810242@qq.com>
2024-06-29 11:18:26 +08:00
Winlin
1f9309ae25
SmartPtr: Support load test for source by srs-bench. v6.0.130 (#4097)
1. Add live benchmark support in srs-bench, which only connects and
disconnects without any media transport, to test source creation and
disposal and verify source memory leaks.
2. SmartPtr: Support cleanup of HTTP-FLV stream. Unregister the HTTP-FLV
handler for the pattern and clean up the objects and resources.
3. Support benchmarking RTMP/SRT with srs-bench by integrating the gosrt
and oryx RTMP libraries.
4. Refine SRT and RTC sources by using a timer to clean up the sources,
following the same strategy as the Live source.

---------

Co-authored-by: Haibo Chen <495810242@qq.com>
Co-authored-by: Jacob Su <suzp1984@gmail.com>
2024-06-21 07:13:12 +08:00
Winlin
e7069788e9
SmartPtr: Support shared ptr for live source. v6.0.129 (#4089)
Detail change log:

1. [Simple,Refactor] Remove member fields of http entry, etc.
e34b3d3aa4
2. [Ignore] Rename source to live_source.
846f95ec96
3. [Ignore] Use directly ptr in consumer.
d38af021ad
4. [Complex, Important] Use shared ptr for live source.
88f922413a

The object relationship:

![live-source](https://github.com/ossrs/srs/assets/2777660/1adb59af-6e7a-40f3-9a4a-1cc849d7dae1)

---

Co-authored-by: Jacob Su <suzp1984@gmail.com>
2024-06-15 07:54:56 +08:00
Winlin
9dba99a1cc
SmartPtr: Support shared ptr for RTC source. v6.0.128 (#4085)
---------

Co-authored-by: Haibo Chen <495810242@qq.com>
2024-06-14 08:07:26 +08:00
Winlin
242152bd6b
SmartPtr: Use shared ptr in RTC TCP connection. v6.0.127 (#4083)
Fix issue https://github.com/ossrs/srs/issues/3784

---

Co-authored-by: Jacob Su <suzp1984@gmail.com>
2024-06-13 16:04:31 +08:00
Jacob Su
1656391c67
RTC: Support dropping h.264 SEI from NALUs. v5.0.213 v6.0.125 (#4057)
try to fix #4052.

---------

Co-authored-by: winlin <winlinvip@gmail.com>
2024-06-03 16:25:49 +08:00
Winlin
7209b73660
WHIP: Fix bug for converting WHIP to RTMP/HLS. v5.0.208 v6.0.113 (#3920)
1. When converting RTC to RTMP, it is necessary to synchronize the audio
and video timestamps. When the synchronization status changes, whether
it is unsynchronized or synchronized, print logs to facilitate
troubleshooting of such issues.
2. Chrome uses the STAP-A packet, which means a single RTP packet
contains SPS/PPS information. OBS WHIP, on the other hand, sends SPS and
PPS in separate RTP packets. Therefore, SPS and PPS are in two
independent RTP packets, and SRS needs to cache these two packets.

---------

Co-authored-by: john <hondaxiao@tencent.com>
2024-02-06 14:06:34 +08:00
winlin
2a2da2253f Switch to 2013-2024. v6.0.109 2024-01-01 10:51:24 +08:00
john
15601b4b2a
RTC: Support OPUS stereo SDP option. v5.0.203 v6.0.105 (#3910)
In an SDK that supports RTC Opus stereo, the parameter "stereo=1" may
appear. SRS (Spatial Reference System) needs to handle this correctly
and return an answer to enable WebRTC stereo support.



---------

`TRANS_BY_GPT4`
2023-12-14 23:29:22 +08:00