Originally posted by Peter Norton on Aug 13, 2014 at 9:34 am
When you are watching a movie online or a recorded show on the DVR and you start fast-forwarding, do you see the scenes whiz by? It allows you to stop at the moment in the video that interests you. This feature is called ‘Trick Play,’ and is implemented on many digital video platforms. The technical implementation is dependent on the type of video streaming and platform in use. On HTTP Live Streaming (HLS) based platforms such as AppleTV, this feature is defined in Technical Note TN2288.
In brief, Trick Play is enabled by creating I-frame playlists, which specify the locations (i.e. byte-ranges) of Intra Frames, also known as I-frames. For more information on I-frame playlists, see the HTTP Live Streaming — Draft.
This summer, as an intern for the Product Development team here at PBS Digital, my main task was to develop a tool that can take an existing HLS video and provide a new version containing I-Frame playlists; thus enabling Trick Play for the video stream.
This effort resulted in PBS’ newest open-source project: https://github.com/pbs/iframe-playlist-generator.
iframe-playlist-generator
To clone the repository:
git clone https://github.com/pbs/iframe-playlist-generator.gitNote: The iframe-playlist-generator module requires an up-to-date installation of the FFmpeg library. At the time of this writing, the module has been tested on FFmpeg version 2.2.2. Earlier releases of FFmpeg were not sufficient.
Usage
The primary usage for this module is to pass in a url or a local path pointing to a variant m3u8 playlist:
playlist_data = update_for_iframes(‘http://devimages.apple.com.edgekey.net/streaming/examples/bipbop_16x9/bipbop_16x9_variant.m3u8’)The update_for_iframes function returns a standard python dictionary containing the generated I-frame playlists and updated master playlist. These can then be used as needed, such as uploading them to an s3 bucket or simply writing them to your local file system. For more information on how to use this module, see the readme on GitHub.
How it was Developed
The process for generating the I-frame playlists is as follows: parse the existing HLS master playlist, parse each variant sub-playlist, evaluate each mpeg-ts segment, use the acquired data to create a byte-range for each I-frame, create the corresponding I-frame playlists, and finally update the original master playlist.
Parsing the playlists
The first two steps are to parse the master playlist to find each sub-playlist, and then to parse each sub-playlist in order to find each individual mpeg-ts segment.
Evaluating the mpeg-ts segments
The next step is to evaluate each mpeg-ts segment and retrieve the needed data from each packet and frame. The data I need includes the position, size, and time of appearance of each I-frame, as well as the position of each video packet. In order to get this data I parse each segment using the tool ffprobe from the FFmpeg library.
Sidenote:
One problem I faced was getting an accurate size for each I-frame. The way I originally decided to calculate this was to take the position of the I-frame and subtract it from the position of the next packet. However, when comparing my calculations to Apple’s sample HLS stream, my calculated sizes were off by 188 bytes (which turns out to be the exact size of an mpeg packet) for every I-frame size. We couldn’t find out why Apple seems to add the size of an extra mpeg packet to each I-frame size, but we decided to mimic their method anyway. So in order to make my calculated I-frame sizes equal to the calculations in Apple’s sample stream, I now add 188 bytes to the total size of each I-frame.
Creating the byte-ranges
Next, using the frame positions and sizes that I’ve calculated, I create a byte-range for each I-frame. This byte-range is what tells the video player where to find the I-frame as well as the size of it. The byte-range format is “packet_size@packet_position” (e.g. “3196@36472”). I also calculate the duration of each I-frame by subtracting its timestamp from the timestamp of the next I-frame.
Creating each I-frame playlist
In order to create each I-frame playlist, I then list these durations, byte-ranges, and the names of the corresponding transport stream files in sequential order.
Updating the master playlist
Finally, in the original master playlist I append pointers to each new I-frame playlist. Each of these pointers include at least a calculated bandwidth, a codec, and a URI to the I-frame playlist. To calculate the bandwidth for each I-frame playlist: total the durations and sizes of every I-frame in the playlist, divide the total bytes by the total duration, and finally multiply by eight to convert to bits. For the codecs string, I simply take the codecs of the corresponding playlist and remove all but the video codec (e.g. “avc1.4d001f, mp4a.40.5” becomes “avc1.4d001f”).
Open-Sourced
The iframe-playlist-generator project has been open-sourced, and PBS invites you to be a part of its continual development. We highly encourage you to clone it from GitHub, play around with it, report issues, make changes, and submit pull requests.
Peter is a rising junior at Franciscan University of Steubenville, majoring in Computer Information Science, and minoring in Marketing and Business Management. He has a passion for software development and for using technology to improve business processes. As an intern at PBS Digital, he learned many skills used in the software development process and built tools for video ingestion and streaming.
