We have discovered that some IP cameras send two publish packets in a
row.
The first packet is flash publish `publish('xxx')`
The second packet is FMLE publish `FCPublish('xxx|@setDataFrame()`
It seems that this is not processed correctly on the SRS side. In fact,
the stream is simply deinitialized, and republish is simply not
supported in this case.
As a fix, I suggest simply ignoring the FMLE publish packet after the
flash publish.
<img width="720" alt="screen"
src="https://github.com/user-attachments/assets/2db806ab-71b9-4e7b-bcf9-c16ea12df671"
/>
Fix issue #4570 by supporting optional `msid` attribute in WebRTC SDP
negotiation, enabling compatibility with libdatachannel and other
clients that don't include msid information.
SRS failed to negotiate WebRTC connections from libdatachannel clients
because:
- libdatachannel SDP lacks `a=ssrc:XX msid:stream_id track_id`
attributes
- SRS required msid information to create track descriptions
- According to RFC 8830, the msid attribute and its appdata (track_id)
are **optional**
If diligently look at the SDP generated by libdatachannel:
```
a=ssrc:42 cname:video-send
a=ssrc:43 cname:audio-send
```
It's deliberately missing the `a=ssrc:XX msid:stream_id track_id` line,
comparing that with this one:
```
a=ssrc:42 cname:video-send
a=ssrc:42 msid:stream_id video_track_id
a=ssrc:43 cname:audio-send
a=ssrc:43 msid:stream_id audio_track_id
```
In such a situation, to keep compatible with libdatachannel, if no msid
line in sdp, SRS comprehensively and consistently uses:
* app/stream as stream_id, such as live/livestream
* type=video|audio, cname, and ssrc as track_id, such as
track-video-video-send-43
This PR adds G.711 (PCMU/PCMA) audio codec support for WebRTC in SRS,
enabling relay-only streaming of G.711 audio between WebRTC clients via
WHIP/WHEP. G.711 is a widely-used, royalty-free audio codec with
excellent compatibility across VoIP systems, IP cameras, and legacy
telephony equipment.
Fixes#4075
Many IP cameras, VoIP systems, and IoT devices use G.711 (PCMU/PCMA) as
their default audio codec. Previously, SRS only supported Opus for
WebRTC audio, requiring transcoding or rejecting G.711 streams entirely.
This PR enables direct relay of G.711 audio streams in WebRTC, similar
to how VP9/AV1 video codecs are supported.
Enhanced WHIP/WHEP players with URL-based codec selection:
```
# Audio codec only
http://localhost:8080/players/whip.html?acodec=pcmuhttp://localhost:8080/players/whip.html?acodec=pcma
# Video + audio codecs
http://localhost:8080/players/whip.html?vcodec=vp9&acodec=pcmuhttp://localhost:8080/players/whep.html?vcodec=h264&acodec=pcma
# Backward compatible (codec = vcodec)
http://localhost:8080/players/whip.html?codec=vp9
```
Testing
```bash
# Build and run unit tests
cd trunk
make utest -j && ./objs/srs_utest
# Test with WHIP player
# 1. Start SRS server
./objs/srs -c conf/rtc.conf
# 2. Open WHIP publisher with PCMU audio
http://localhost:8080/players/whip.html?acodec=pcmu
# 3. Open WHEP player to receive stream
http://localhost:8080/players/whep.html
```
## Related Issues
- Fixes#4075 - WebRTC G.711A Audio Codec Support
- Related to #4548 - VP9 codec support (similar relay-only pattern)
This PR adds separate audio and video frame counting to the HTTP API
(`/api/v1/streams/`) for better stream observability. The API now
reports three frame fields:
- `frames` - Total frames (video + audio)
- `video_frames` - Video frames/packets only
- `audio_frames` - Audio frames/packets only
This enhancement provides better visibility into stream composition and
helps detect issues with CBR/VBR streams, audio/video sync problems, and
codec-specific behavior.
**Before:**
```json
{
"streams": [
{
"frames": 0, // video frames.
}
]
}
```
**After:**
```json
{
"streams": [
{
"frames": 6912, // video frames.
"audio_frames": 5678, // audio frames.
"video_frames": 1234, // video frames.
}
]
}
```
Frame Counting Strategy
- All protocols report frames every N frames to balance accuracy and
performance
- Frames are counted at the protocol-specific message/packet level:
- RTMP: Counts RTMP messages (video/audio)
- WebRTC: Counts RTP packets (video/audio)
- SRT: Counts MPEG-TS messages (H.264/HEVC/AAC)
for issue #4418, #4151, #4076 .DVR Missing First Few Seconds of
Audio/Video
### Root Cause
When recording WebRTC streams to FLV files using DVR, the first 4-6
seconds of audio/video are missing. This occurs because:
1. **Packets are discarded before A/V sync is available**: The
RTC-to-RTMP conversion pipeline actively discards all RTP packets when
avsync_time <= 0.
2. **Original algorithm requires 2 RTCP SR packets**: The previous
implementation needed to receive two RTCP Sender Report (SR) packets
before it could calculate the rate for audio/video synchronization
timestamp conversion.
3. **Delay causes packet loss**: Since RTCP SR packets typically arrive
every 2-3 seconds, waiting for 2 SRs means 4-6 seconds of packets are
discarded before A/V sync becomes available.
4. **Audio SR arrives slower than video SR**: As reported in the issue,
video RTCP SR packets arrive much faster than audio SR packets. This
asymmetry causes audio packets to be discarded for a longer period,
resulting in the audio loss observed in DVR recordings.
### Solution
1. **Initialize rate from SDP**: Use the sample rate from SDP (Session
Description Protocol) to calculate the initial rate immediately when the
track is created.
Audio (Opus): 48000 Hz → rate = 48 (RTP units per millisecond)
Video (H.264/H.265): 90000 Hz → rate = 90 (RTP units per millisecond)
2. **Enable immediate A/V sync:** With the SDP rate available,
cal_avsync_time() can calculate valid timestamps from the very first RTP
packet, eliminating packet loss.
3. **Smooth transition to precise rate**: After receiving the 2nd RTCP
SR, update to the precisely calculated rate based on actual RTP/NTP
timestamp mapping.
## Configuration
Added new configuration option `init_rate_from_sdp` in the RTC vhost
section:
```nginx
vhost rtc.vhost.srs.com {
rtc {
# Whether initialize RTP rate from SDP sample rate for immediate A/V sync.
# When enabled, the RTP rate (units per millisecond) is initialized from the SDP
# sample rate (e.g., 90 for video 90kHz, 48 for audio 48kHz) before receiving
# 2 RTCP SR packets. This allows immediate audio/video synchronization.
# The rate will be updated to a more precise value after receiving the 2nd SR.
# Overwrite by env SRS_VHOST_RTC_INIT_RATE_FROM_SDP for all vhosts.
# Default: off
init_rate_from_sdp off;
}
}
```
**⚠️ Important Note**: This config defaults to **off** because:
- ✅ When **enabled**: Fixes the audio loss problem (no missing first 4-6
seconds)
- ❌ When **enabled**: VLC on macOS cannot play the video properly
- ✅ Other platforms work fine (Windows, Linux)
- ✅ FFplay works fine on all platforms
Users experiencing audio loss in DVR recordings can enable this option
if they don't need VLC macOS compatibility. We're investigating the VLC
macOS issue to make this feature safe to enable by default in the
future.
---------
Co-authored-by: winlin <winlinvip@gmail.com>
Co-authored-by: OSSRS-AI <winlinam@gmail.com>
Regression since 20f6cd595c
The early code might meet bridge is empty when
there is no bridge(e.x. rtc to rtc). Then srs_freep will free the brige.
Remove this code that seems redundant.
---------
Co-authored-by: Jacob Su <suzp1984@gmail.com>
Signed-off-by: Jack Lau <jacklau1222@qq.com>
This PR refactors the stream bridge architecture in SRS to improve code
organization, type safety, and maintainability by replacing the generic
ISrsStreamBridge interface with protocol-specific bridge classes and
target interfaces.
1. New Target Interface Architecture:
- Introduces ISrsFrameTarget for AV frame consumers (RTMP sources)
- Introduces ISrsRtpTarget for RTP packet consumers (RTC sources)
- Introduces ISrsSrtTarget for SRT packet consumers (SRT sources)
2. Protocol-Specific Bridge Classes:
- SrsRtmpBridge: Converts RTMP frames to RTC/RTSP protocols
- SrsSrtBridge: Converts SRT packets to RTMP/RTC protocols
- SrsRtcBridge: Converts RTC packets to RTMP protocol
3. Simplified Bridge Management:
- Removes the generic SrsCompositeBridge chain pattern
- Each source type now uses its appropriate bridge type directly
With this improvement, you are able to implement very complex bridge and
protocol converting, for example, you can bridge RTMP to RTC with opus
audio when you support enhanced RTMP with opus.
Another plan is to support bridging RTC to RTSP, directly without
converting RTP to media frame packet, but directly deliver RTP packet
from RTC source to RTSP source.
---------
Co-authored-by: OSSRS-AI <winlinam@gmail.com>
This PR introduces anonymous coroutine macros for easier coroutine
creation and improves the State Threads (ST) mutex and condition
variable handling in SRS.
- **Added coroutine macros**: `SRS_COROUTINE_GO`, `SRS_COROUTINE_GO2`,
`SRS_COROUTINE_GO_CTX`, `SRS_COROUTINE_GO_CTX2`
- **Added `SrsCoroutineChan`**: Channel for sharing data between
coroutines with coroutine-safe operations
- **Simplified coroutine creation**: Go-like syntax for creating
anonymous coroutines with code blocks
---------
Co-authored-by: Jacob Su <suzp1984@gmail.com>
Co-authored-by: OSSRS-AI <winlinam@gmail.com>
WebRTC RTC publish streams use timer callbacks (`SrsRtcPublishRtcpTimer`
and `SrsRtcPublishTwccTimer`) that can cause race conditions in SRS's
coroutine-based architecture. The timer callbacks are heavy functions
that may trigger coroutine switches, during which the timer object can
be freed by another coroutine, leading to use-after-free crashes.
The race condition occurs because:
1. Timer callbacks (`on_timer`) perform heavy operations that can yield
control
2. During coroutine switches, other coroutines may destroy the timer
object
3. When control returns, the callback continues executing on a freed
object
Fixes potential crashes in WebRTC RTC publish streams under high
concurrency.
Move global xpps statistics variables from `srs_app_server.cpp` to
`srs_kernel_kbps.cpp`.
Extract global shared timers from `SrsServer` into new `SrsSharedTimer`
class.
Extract WebRTC session management logic from `SrsServer` into dedicated
`SrsRtcSessionManager` class.
Extract PID file handling into dedicated `SrsPidFileLocker` class.
---------
Co-authored-by: OSSRS-AI <winlinam@gmail.com>
This PR consolidates the SRT and RTC server functionality into the main
SrsServer class, eliminating the separate `SrsSrtServer` and
`SrsRtcServer` classes and their corresponding adapter classes. This
architectural change simplifies the codebase by removing the hybrid
server pattern and integrating all protocol handling directly into
`SrsServer`.
As unified connection manager (`_srs_conn_manager`) for all protocol
connections, all incoming connections are checked against the same
connection limit in `on_before_connection()`. This enables consistent
connection limits: `max_connections` now protects against resource
exhaustion from any protocol, not just RTMP.
Remove modules because it's not used now, so only keep the server
application module and main entry point. Remove the wait group to run
server, instead, directly run server and invoke the cycle method.
After this PR, the startup workflow and servers architecture should be
much easier to maintain.
---------
Co-authored-by: OSSRS-AI <winlinam@gmail.com>
This PR introduces a comprehensive stream publish token system that
prevents race conditions when multiple publishers attempt to publish to
the same stream URL simultaneously across different protocols (RTMP,
WebRTC, SRT).
* Race Condition Issue: Multiple publishers could create duplicate
sources for the same stream when context switches occurred during source
initialization in SRS's coroutine-based architecture
* Cross-Protocol Conflicts: Different protocols (RTMP, RTC, SRT) could
simultaneously publish to the same stream URL without coordination
* Resource Management: No centralized mechanism to ensure exclusive
stream publishing access
---------
Co-authored-by: OSSRS-AI <winlinam@gmail.com>
This PR fixes a critical race condition in SRS source managers where
multiple coroutines could create duplicate sources for the same stream.
- **Atomic source creation**: Source lookup, creation, and pool
insertion now happen atomically within lock scope
- **Consistent interface**: Standardize on `ISrsRequest*` interface
throughout codebase
- **Handler simplification**: Remove `ISrsLiveSourceHandler*` parameter,
obtain from global server instance
---------
Co-authored-by: OSSRS-AI <winlinam@gmail.com>
This PR makes WebRTC a core feature of SRS and enforces C++98
compatibility by:
1. Always Enable WebRTC Support
- Remove `--rtc=on|off` configuration option - WebRTC is now always
enabled
- Eliminate all `#ifdef SRS_RTC` conditional compilation blocks
- Include WebRTC-related modules (RTC, SRTP, DTLS) in all builds
- Update build scripts to always link WebRTC dependencies
2. Enforce C++98 Compatibility
- Remove `--cxx11=on|off` and `--cxx14=on|off` configuration options
- Force `SRS_CXX11=NO` and `SRS_CXX14=NO` in build system
- Move these options to deprecated section with warnings
- Ensure codebase maintains C++98 standard compatibility
3. Remove Windows/Cygwin Support
- Remove all Windows and Cygwin64 conditional compilation blocks (#ifdef
_WIN32, #ifdef CYGWIN64)
- Delete Cygwin64 build configurations from build scripts (
auto/options.sh, auto/depends.sh, configure)
- Remove Cygwin64 assembly files and State Threads platform support (
md_cygwin64.S)
- Eliminate Windows-specific GitHub Actions workflows and CI/CD jobs
- Remove NSIS packaging files and Windows installer generation
- Delete Windows documentation and update feature lists to mark support
as removed in v7.0
- Simplify OS detection to only support Unix-like systems (Linux, macOS)
4. Code Cleanup
- Remove conditional WebRTC code blocks throughout the codebase
- Simplify build configuration by removing WebRTC-related conditionals
- Update constructor delegation patterns to be C++98 compatible
- Fix vector initialization to use C++98 syntax
- Eliminate Windows-specific implementations for file operations, time
handling, and networking
- Unified platform handling with consistent POSIX API usage
---------
Co-authored-by: OSSRS-AI <winlinam@gmail.com>
This PR removes the multi-threading infrastructure from SRS and
consolidates the codebase to use single-thread architecture exclusively.
This is a architectural simplification that aligns with SRS's
coroutine-based design philosophy.
* Simplified Architecture: Eliminates complexity of multi-threading
coordination
* Better Alignment: Matches SRS's coroutine-based single-thread design
philosophy
* Reduced Complexity: Removes potential race conditions and threading
bugs
* Cleaner Code: More focused modules with clear responsibilities
* Easier Maintenance: Fewer moving parts and clearer execution flow
---------
Co-authored-by: OSSRS-AI <winlinam@gmail.com>
This PR refactors the HTTP hooks system from static methods to a proper
interface-based architecture, improving code maintainability,
testability, and extensibility.
1. **Testability**: Interface allows easy mocking for unit tests
1. **Extensibility**: Custom hook implementations can be injected
1. **Maintainability**: Clear separation of concerns and better code
organization
1. **Documentation**: Comprehensive inline documentation for all hook
methods
1. **Future-proofing**: Enables plugin architecture and custom hook
handlers
---------
Co-authored-by: OSSRS-AI <winlinam@gmail.com>
**Introduce**
This pull request builds upon the foundation laid in
https://github.com/ossrs/srs/pull/4289 . While the previous work solely
implemented unidirectional HEVC support from RTMP to RTC, this
submission further enhances it by introducing support for the RTC to
RTMP direction.
**Usage**
Launch SRS with `rtc2rtmp.conf`
```bash
./objs/srs -c conf/rtc2rtmp.conf
```
**Push with WebRTC**
Upgrade browser to Chrome(136+) or Safari(18+), then open [WHIP
encoder](http://localhost:8080/players/whip.html?schema=http&&codec=hevc),
push stream with URL that enables HEVC by query string `codec=hevc`:
```bash
http://localhost:1985/rtc/v1/whip/?app=live&stream=livestream&codec=hevc
```
This query string `codec=hevc` is used to select the video codec, and
generate lines in the answer SDP.
```
m=video 9 UDP/TLS/RTP/SAVPF 49 123
a=rtpmap:49 H265/90000
```
The encoder log also show the codec:
```
Audio: opus, 48000HZ, channels: 2, pt: 111
Video: H265, 90000HZ, pt: 49
```
**Play with RTMP**
Play HEVC stream via RTMP.
```bash
ffplay -i rtmp://localhost/live/livestream
```
You will see the codec in logs:
```
Stream #0:0: Audio: aac (LC), 48000 Hz, stereo, fltp
Stream #0:1: Video: hevc (Main), yuv420p(tv, bt709), 320x240, 30 fps, 30 tbr, 1k tbn
```
You can also use [WHEP
player](http://localhost:8080/players/whep.html?schema=http&&codec=hevc)
to play the stream.
Important refactor with AI:
* [AI: Refactor packet cache for RTC frame
builder.](b8ffa1630e)
* [AI: Refactor the packet copy and free for
SrsRtcFrameBuilder](f3487b45d7)
* [AI: Refactor the frame detector for
SrsRtcFrameBuilder](4ffc1526b9)
* [AI: Refactor the packet_video_rtmp for
SrsRtcFrameBuilder](81f6aef4ed)
* [AI: Add utests for
SrsCodecPayload.codec](61eb1c0bfc)
* [AI: Add utests for VideoPacketCache in
SrsRtcFrameBuilder.](fd25480dfa)
* [AI: Add utests for VideoFrameDetector in
SrsRtcFrameBuilder.](b4aa977bbd)
* [AI: Add regression test for RTC2RTMP with
HEVC.](5259a2aac3)
---------
Co-authored-by: Jacob Su <suzp1984@gmail.com>
Co-authored-by: winlin <winlinvip@gmail.com>
