Adaptive Bitrate Streaming

Adaptive bitrate streaming or ABR streaming, sometimes shortened to ABS, is a technique for dynamically adjusting a stream’s compression level and video quality to match bandwidth availability.

What is a Bitrate for Streaming?

The video bitrate of a stream refers to the speed for transferring video data to a user’s device. The standard measure for a bitrate for streaming video is megabits per second (Mbps), which differs from a video’s file size measured in megabytes (MBs).

In general, a higher bitrate means the video quality will be better, but if it exceeds a user’s bandwidth — also measured in Mbps — then buffering can occur. Buffering means the video player cannot download the video file fast enough to keep the video playing at normal speed.

These are some factors that impact the streaming video bitrate:

• Video resolution: The resolution of a video refers to the number of pixels within a frame. Most video streams have a resolution that’s 480, 720, 1080, or 4K. A higher resolution makes images sharper and may be necessary to have a quality viewing experience on larger displays.
• Frame rate: Most broadcasters use 24 frames per second (fps), but sports events or other streams that require a smoother playback may use 30 to 60 fps. A higher frame rate can display motion better, but more frames also require additional data.
• Codec: A higher quality video — with a greater resolution and frame rate — generally has a higher bitrate. That said, some codecs that compress video data are more efficient than others. Newer codecs like H.264 and H.265 result in a higher quality video at lower bitrates than legacy codecs.

What is Adaptive Playback?

Adaptive bitrate streaming (ABR) is an algorithm where a video player client decides which bitrate segments to download. The result is that broadcasters can offer multiple streams with different bitrates to users. The video player detects fluctuations of the internet connection and the processing capacity of the user’s device to automatically upgrade or downgrade the stream’s bitrate in real-time.

Adaptive playback, therefore, is a way for video players to dynamically match a stream’s quality to network conditions, available bandwidth, and device performance.

How Does ABR Streaming Work?

1. Video Preparation

First, the video content needs to be prepped for ABR live streaming. An encoder needs to encode the video into several bitrates depending on the target streaming profile. Most online video platforms are capable of transcoding videos into multiple formats from a single source file.

Each of these encoded video files needs to be segmented into multi-second parts. The size of these video chunks varies depending on the implementation, but they’re generally between 2 and 10 seconds each. This approach works well with HTTP-based streaming protocols that progressively download video content.

2. Initial Startup

Before a video player can begin playback, it downloads a manifest file that describes the video chunks and bitrates available. This manifest file — the .m3u8 playlist for HLS and media presentation description (MPD) for DASH — contains the information that makes up a streaming profile, which we’ll cover later on.

The video player³ can simply use the manifest as a guide to request the video assets that are most suitable for the user’s device and connection. Then, it begins playback immediately when there’s enough media content available in the local buffer.

The benefit of streaming video over HTTP is that the content can be stored and delivered from an ordinary web server. That means ABR works well with content delivery networks (CDNs) to further reduce latency, lower packet loss, and increase delivery speed.

3. Dynamic Playback

Video players usually start streaming at the lowest bit rate and request higher or lower quality video chunks as network conditions change. Each player has its own ABR algorithm for deciding which bitrates to download next. There are two common types of selection algorithms that video players use: throughput and buffer.

Throughput-based algorithms measure the download speed of previous video chunks to determine which bitrate to choose next. The problem is that a throughput estimate may not match the encoder bitrate during live streams.

Buffer-based algorithms attempt to control the buffer occupancy to ensure there’s always enough video ready for playback. If the local buffer is running out of media, the next bitrate will be lower to keep ahead of playback. The problem is knowing how far to lower the bitrate to fill the buffer without downgrading quality too much.

Some adaptive players even offer a hybrid of the two. Over the coming years, more advanced algorithms using machine learning may gain traction as well.

Adaptive Bitrate Streaming Profiles

A streaming profile is the set of variations that you want to make available for a particular video. This means creating a ladder that includes different bitrates, resolutions, codecs, and more.

Most ABR profiles are universal ladder designs meant to optimize all video files for any streaming device and network condition. Apple even proposed a fixed bitrate encoding ladder (see below), which Netflix later refined to maximize video quality at each particular rung.

Another streaming profile strategy gaining popularity is a context-aware encoding (CAE), which considers user devices when encoding as well. Each piece of content gets encoded for specific device types — from smartphones to smart TVs — to build a custom bitrate ladder. This enables broadcasters to offer a higher Quality of Experience with reduced bandwidth requirements.

In the end, there’s not a single streaming profile that fits every broadcaster’s needs.

The Benefits of Adaptive Playback

The primary benefit of adaptive bitrate streaming is an improved Quality of Experience. Broadcasters can offer the highest quality video possible for their viewers without risking buffering or other interruptions to the experience.

Along with smoother streams, ABR offers a faster startup. Since ABR streams usually start with a low bitrate stream until the selection algorithm estimates bandwidth capacity or buffering occupancy, the video can begin playback immediately. That means brands don’t need to worry about losing viewers due to slow loading times.

ABR adaptive bitrate streaming also improves the viewing experience on smartphones and tablets. In the past, data plan limitations and low device processing power made streaming video on the go a challenge for broadcasters. ABR ensures mobile users can watch video streams without buffering despite these potential limitations.

Since ABR streams utilize HTTP for delivery, it works with most web servers and CDNs. That means it’s cheaper than setting up specialized servers or keeping persistent connections open. Using ABR streaming, therefore, can be a cost-effective way to scale while maintaining a high-quality streaming experience.

HTTP-Based Adaptive Protocols

Streams deployed over HTTP are not technically “streams.” Rather, they’re progressive downloads sent via regular web servers. Using adaptive bitrate streaming, HTTP-based protocols deliver the best video quality and viewer experience possible — no matter the connection, software, or device. Some of the most common HTTP-based protocols include MPEG-DASH and Apple’s HLS.

• HTTP Live Streaming (HLS)

HLS is an HTTP-based adaptive bitrate streaming communications protocol developed by Apple Inc. and released in 2009. Support for the protocol is widespread in media players, web browsers, mobile devices, and streaming media servers. As of 2019, an annual video industry survey has consistently found it to be the most popular streaming format.

HLS resembles MPEG-DASH in that it works by breaking the overall stream into a sequence of small HTTP-based file downloads, each downloading one short chunk of an overall potentially unbounded transport stream. A list of available streams, encoded at different bit rates, is sent to the client using an extended M3U playlist.

Video Codecs: H.265, H.264

Audio Codecs: AAC-LC, HE-AAC+ v1 & v2, xHE-AAC, Apple Lossless, FLAC

Playback Compatibility: Great (All Google Chrome browsers; Android, Linux, Microsoft, and macOS devices; several set-top boxes, smart TVs, and other players)

Benefits: Adaptive bitrate and widely supported

Drawbacks: Quality of experience is prioritized over low latency

Latency: 6–30 seconds (lower latency only possible when tuned)

Variant Formats: Low-Latency HLS (see below), PHLS (Protected HTTP Live Streaming)

• Dynamic Adaptive Streaming over HTTP (DASH)

Also known as MPEG-DASH, is an adaptive bitrate streaming technique that enables high-quality streaming of media content over the Internet delivered from conventional HTTP web servers.

Similar to Apple’s HTTP Live Streaming (HLS) solution, MPEG-DASH works by breaking the content into a sequence of small segments, which are served over HTTP. Each segment contains a short interval of playback time of content that is potentially many hours in duration, such as a movie or the live broadcast of a sports event. The content is made available at a variety of different bit rates, i.e., alternative segments encoded at different bit rates covering aligned short intervals of playback time. While the content is being played back by a MPEG-DASH client, the client uses a bit rate adaptation (ABR) algorithm to automatically select the segment with the highest bitrate possible that can be downloaded in time for playback without causing stalls or re-buffering events in the playback.

Video Codecs: Codec-agnostic

Audio Codecs: Codec-agnostic

Playback Compatibility: Good (All Android devices; most post-2012 Samsung, Philips, Panasonic, and Sony TVs; Chrome, Safari, and Firefox browsers)

Benefits: Vendor independent, international standard for adaptive bitrate

Drawbacks: Not supported by iOS or Apple TV

Latency: 6–30 seconds (lower latency only possible when tuned)

Variant Formats: MPEG-DASH CENC (Common Encryption)

• Microsoft Smooth Streaming

Microsoft developed Microsoft Smooth Streaming in 2008 for use with Silverlight player applications. It enables adaptive delivery to all Microsoft devices. The protocol can’t compete with other HTTP-based formats and isn’t commonly used. In a Video Streaming Report by Wowzaless than 3 per cent of respondents were using Smooth Streaming.

Video Codecs: H.264, VC-1

Audio Codecs: AAC, MP3, WMA

Playback Compatibility: Good (Microsoft and iOS devices, Xbox, many smart TVs, Silverlight player-enabled browsers)

Benefits: Adaptive bitrate and supported by iOS

Drawbacks: Proprietary technology

Latency: 6–30 seconds (lower latency only possible when tuned)

Emerging Technologies

New technologies like WebRTC and SRT promise to change the landscape. Similar to low-latency CMAF for DASH and Apple Low-Latency HLS, these protocols were designed with latency in mind.

• Low-Latency CMAF for DASH

Low-latency CMAF for DASH is another emerging technology for speeding up HTTP-based video delivery. Although it’s still in its infancy, the technology shows promise in delivering superfast video at scale by using shorter data segments. That said, many vendors have prioritized support for Low-Latency HLS over that of low-latency CMAF for DASH.

Playback Compatibility: Any players that aren’t optimized for low-latency CMAF for DASH can fall back to standard (higher-latency) DASH behaviour

Benefits: Low latency meets HTTP-based streaming

Drawbacks: As an emerging spec, vendors are still implementing support

Latency: 3 seconds or less

• Low-Latency HLS

Low-Latency HLS (LL-HLS) is the latest and greatest technology when it comes to low-latency streaming. The proprietary protocol promises to deliver sub-three-second streams globally. It also offers backward compatibility to existing clients.

Playback Compatibility: Any players that aren’t optimized for Low-Latency HLS can fall back to standard (higher-latency) HLS behaviour

HLS-compatible devices include MacOS, Microsoft, Android, and Linux devices; all Google Chrome browsers; several set-top boxes, smart TVs, and other players

Benefits: Low latency, scalability, high quality, and backward compatibility.

Drawbacks: As an emerging spec, vendors are still implementing support

Latency: 2 seconds or less

• SRT

This open-source protocol is recognized as a proven alternative to proprietary transport technologies — helping to deliver reliable streams, regardless of network quality. It competes directly with RTMP and RTSP as a first-mile solution, but it’s still being adopted as encoders, decoders, and players add support.

From recovering lost packets to preserving timing behaviour, SRT was designed to solve video contribution and distribution challenges across the public internet. And it’s quickly taking the industry by storm. One interactive use case for which SRT proved instrumental was the 2020 virtual NFL draft. The NFL used this game-changing technology to connect 600 live feeds for the first entirely virtual event.

Video Codecs: Codec-agnostic

Audio Codecs: Codec-agnostic

Playback Compatibility: Limited (VLC Media Player, FFPlay, Haivision Play Pro, Haivision Play, Larix Player, Brightcove)

Benefits: High-quality, low-latency video over suboptimal networks

Drawbacks: Not widely supported for video playback

Latency: 3 seconds or less, tunable based on how much latency you want to trade for packet loss

• WebRTC

WebRTC delivers near-instantaneous voice and video streaming to and from any major browser as the speediest technology available. The framework was designed for pure chat-based applications, but it’s now finding its way into more diverse use cases.

Scalability remains a challenge with WebRTC.

Video Codecs: H.264, VP8, VP9

Audio Codecs: Opus, iSAC, iLBC

Playback Compatibility: Chrome, Firefox, and Safari support WebRTC without any plugin

Benefits: Super fast and browser-based

Drawbacks: Designed for video conferencing and not scale

Latency: Sub-500-millisecond delivery