Streaming Video Playback on the Web
At BoxCast, we take control of the end-to-end video streaming solution for live and on-demand, which allows us to optimize the user experience, quality, and performance of the video capture and playback at each step.
We think of the streaming video lifecycle to include:
- Encoding: Hardware/software to take frames from a camera and get them ready to send to cloud.
- Transferring: Protocols for sending data up to the cloud and dealing with network shenanigans.
- Producing: Adding video overlays, multiple camera angles, etc.
- Transcoding: Make it so the video can play on any device on any network.
- Delivering: Scale the video content distribution so it can reach mass audiences across the world.
- Presenting: Immerse the viewer in the video playback experience.
- Monetizing: Advertisements, pay-per-view, etc. as necessary to support your business.
Let’s focus on Presenting the video for a moment.
There have been many tools for playing back video in the web browser over the years. Flash ruled the day for a long time because native browser playback was sketchy at best. Now, in 2016, browsers have come a long way and the playing field looks a little different.
Adaptive Bitrate Streaming With HLS
Unfortunately, sending a high quality, reliable video stream to a large audience isn’t as simple as uploading an MP4 file on your web server and instructing a web page to play it. That wouldn’t work because mobile devices on weak networks ought to see a lower quality video than a PC on a good high speed internet — not to mention that it just doesn’t make sense for live video.
Instead, we utilize HLS (HTTP Live Streaming) for delivering our video. HLS is simple in principle — the video gets broken into a bunch of little chunks, and the server sends a few different playlists to the client suggesting which order to play the chunks on based on the bandwidth the client can support. These are all served over HTTP which allows us to leverage our massive CDN (Content Distribution Network) infrastructure to scale to millions of viewers. HLS rose to popularity because of its native support on iPhones (it was Apple’s invention after all) and is really a solid and straightforward way of offering a robust stream that works on all networks.
iOS and most newer Android devices support HLS playback natively. You tell the web browser to play it with a simple <video> tag, and it happily does. The same follows for newer Safari and Edge web browsers. But this is not true of Chrome, Firefox, and IE.
Traditionally, playing HLS streams (and many other video formats for that matter) in browsers without native support meant utilizing Flash. But Flash playback is wrought with issues (security and performance to start).
Media Source Extensions
Media Source Extensions (MSE) is a newer browser technology available in Chrome, Safari (desktop only), Firefox, and IE11.
In 2015, we saw major streaming providers (YouTube, Netflix, etc.) take a leap and prefer MSE for playback. They were able to justify it because viewers have steadily continued to adopt modern browsers:
Let’s Get Technical For a Bit
The way MSE works in practice for HLS is pretty neat:
- We then fetch the subsequent bandwidth-specific .m3u8 file that matches what we calculate to be reasonable.
- We parse that to find a list of .ts segments.
- We fetch several of those segments in order, creating a buffer to allow smooth playback.
- As those are segments are fetched into the buffer, we transmux them into MP4 fragments.
- We then feed those fragment into a <video> tag as if it were MP4 video all along.
Impressions of the MSE-based Player
It’s been surprising to all of us to watch BoxCast live church streams and sporting events side-by-side on our legacy Flash player vs. our new MSE based player. We couldn’t have hoped for a better outcome. The exact same source video is much smoother and has less of an impact on our computer resources. The new BoxCast v3 player is being slowly rolled out to customers as we speak, and we’ll likely retire the legacy player in the coming months.
Obligatory Note About Our Startup