The second release of Elixir WebRTC
It’s been three months since we posted about the initial release of Elixir WebRTC. In that time, we’ve added a lot of features, made a bunch of improvements, and now we are about to release the second version of the library. Here, I’m going to tell you about what exactly we have done.
WebRTC improvements and new features
Here’s an overview of the new WebRTC-specific features that we have added to version 0.2:
- Support for different types of RTCP packets (these carry various information about the media stream, contrary to the RTP packets, which carry the media itself):
- Transport Wide Congestion Control: our PeerConnection can create these based on incoming RTP packets (what are the delays between packets, which packets have been lost?) and send them to the other peer, allowing adjustments to stream parameters (like encoder’s bitrate settings) to better match network conditions. Without this, for example, Chrome will cap the video at a lower quality.
- NACK: negative acknowledgments, our PeerConnection will send these when it detects lost packets. Then the other side can proceed with the retransmissions.
- PLI: simplifying a bit, allows our PeerConnection to request a new keyframe from the remote peer. - RTX (retransmissions): our PeerConnection can not only request retransmissions but also handle remote NACKs and retransmit packets itself.
- ICE features:
- support for mDNS candidates: these are used to conceal private IP addresses.
- TURN support: our PeerConnection can generate relay candidates and use a TURN server (previously, we could only handle remote relay candidates).
- tracking remote keepalives, so our PeerConnection can properly detect when the remote peer was disconnected.
Broadcaster
We also created a WHIP/WHEP broadcasting server based on Elixir WebRTC called Broadcaster (not very original, I know). It allows for streaming media live from a WHIP source (whether it’s an OBS or a web browser) to multiple WHEP clients, typically web browsers. The selling point of WHIP/WHEP broadcasting is the very low latencies — we can keep them much lower than, for example, using OBS for streaming to YouTube.
Initially, Broadcaster was supposed to be just another example app (alongside Reco, which we presented in the last blog post), but we used it to stream the latest ECK Meetup, and it was a success. We decided to go with the flow and also stream some other upcoming events. If you are organizing an event and would like to use Broadcaster to stream it, reach out to us!
Benchmarks and performance
The work on benchmarking has started — we are not 100% sure of the results yet, but here’s a little sneak peek into that.
We created a simple benchmarking tool that allows us to test the transmission between two PeerConnections using an arbitrary number of tracks and bitrate. In the presented scenarios, we used about 1.6 mbit/s per track (200 packets/s, 1000 bytes of payload each, which mimics higher-quality video).
My MacBook M1 Pro was able to handle more than 200 tracks sent from one peer to the other peer (one direction only) and an AMD EPYC 7502P 32-core machine was able to handle about 100 tracks, utilizing less than 5 cores (even more in cases when the number of tracks was split between the peers, e.g. 50 tracks in both directions, as generally, the sender was the bottleneck).
It’s worth mentioning that the level of parallelization of a single PeerConnection is quite low at the moment (it only uses 3 OTP processes, for ICE, DTLS/encryption, and everything else), so the clock speed of a processor is much more important than the number of cores, but that hopefully will change, when we get to work on splitting the load of particular transceivers to different processes. In general, the possibilities for optimization are huge.
Remember that this is only a single PeerConnection, in a typical videoconferencing scenario you’d have a PeerConnection per participant. Even in big meetings (>100 participants) a single user usually receives 20–30 tracks (or how many video tiles an app can fit in a single view) simultaneously, and at that point, the performance of a single PeerConnection probably won’t be the issue to care about. Because of that, this metric is not immediately useful, but you can create a rough estimation of how the Elixir WebRTC could perform when used in the videoconferencing scenario e.g. as an SFU.
Plans for the future
We still have a lot to do, these are the things that we are planning to tackle before the third release:
- Adding support for Simulcast.
- More performance benchmarking and profiling.
- API improvements, especially in the area of codec configuration.
- Experimental support for Elixir WebRTC in the J̶e̶l̶l̶y̶f̶i̶s̶h̶ FishJam Multimedia Server (another project that we are working on and soon will go through rebranding).
- Further improvements to the Broadcaster app.
If you’re interested in Elixir WebRTC and want to keep track of the new features and changes, take a look at the roadmap. Also, check out the upcoming CommCon in London — we’ll be there to speak about Elixir WebRTC.
Thanks for reading!
