Streaming UHD/HDR content in the browser

We wanted to explore the options to watch HDR content in a web browser and understand what needs to be done to achieve that so we could add this to our streaming lab. This is what we have learnt so far.

First of all there are currently two main codecs that efficiently encodes and decodes HDR (Ultra HD) content and that is HEVC and VP9. Looking at what the latest browsers support we know that HEVC is only supported on Microsoft Edge on a platform where HEVC hardware decoder exists. VP9 on the other hand is supported in both Chrome and Firefox so we decided to explore that option.

From HEVC/MP4 to VP9/WEBM

At hand we had a trailer in HDR that was encoded with HEVC and stored in a regular MP4 container, so the first step is to transcode this to VP9. As we wanted to prepare this for adaptive HTTP streaming we would also have to package this into an HTTP streaming format and we decided to choose MPEG-DASH as the format. One question to answer was whether it is possible to put the video and audio into chunked ISOBMFF containers. We tested with Amazon Elastic Transcoder and it can transcode from HEVC to VP9 but not at the same time store it in an ISOBMFF container. Next we tried with ffmpeg and it turns out that the functionality to store VP9 in MP4 containers is on a very experimental level. So the answer was to simply stick with webm containers which is something MPEG-DASH also supports.

To prepare the MPEG-DASH package we took the HEVC encoded file and transcoded it into several VP9 encoded files with different bitrates and where we also separated audio and video. This can be achieved with the following ffmpeg line for each bitrate and resolution.

`$ ffmpeg -i exodus-hdr.mp4 -c:v libvpx-vp9 -s 1920x1080 -b:v 2000k -keyint_min 60 -g 60 -tile-columns 4 -frame-parallel 1 -an -f webm -dash 1 exodus_1920x1080_2000k.webm`

To easier be able to automate this part we also built a small Python wrapper around ffmpeg that we called vp9-dash. It is also available in our Open Source Software Center on Github.

$ vp9-dash Exodus_UHD_HDR.mp4 Exodus

So now we have 3 versions with different bitrate and resolution and 1 audio version encoded in VP9 and stored in WEBM container.

Creating MPEG-DASH for WEBM

According to ffmpeg documentation it should be possible to also build an MPEG-DASH manifest but we never got that part to work. Instead we tried to find another tool that could do this. What we discovered is that most of the MPEG-DASH packagers are quite centered around fragmented MP4 except for Google’s open source package shaka-packager that you can find here. And it was quite handy that they have built a docker container with everything pre-installed so we only had to pull the image from the docker registry and could run it on our docker container host in the lab.

docker run -v /mnt/s3/webm/:/mnt/ -it — rm google/shaka-packager

and then with this command we could create the MPEG-DASH manifest file we needed.

packager input=video_Exodus_1920x1080_2000k.webm,stream=video,output=v-Exodus-2000.webm input=video_Exodus_1280x720_1200k.webm,stream=video,output=v-Exodus-1200.webm input=video_Exodus_960x540_800k.webm,stream=video,output=v-Exodus-800.webm input=audio_Exodus_128k.webm,stream=audio,output=a-Exodus-128.webm — mpd_output exodus.mpd

Now that we have the MPEG-DASH manifest and the VP9/WEBM files we wanted we just had to put it in the right S3 bucket in the streaming lab and we could try it out in our Shaka Player based MPEG-DASH player.

You could try it out yourself and the stream is available here: http://lab.eyevinn.technology/webm/exodus.mpd

It is left to verify that we have not lost any HDR metadata on the way and find a process to automate this a bit more. Maybe a combination of docker containers and Lambda functions. An idea to be tested later.

If you want to know more about this or just want to connect with me you can find me on Twitter (@JonasBirme) or in our Streaming Tech Sweden community on Slack where you also will find some of my colleagues who worked with this (Magnus Svensson and Johan Skaneby).

Jonas Birmé is a Solution Architect and partner at Eyevinn Technology. A Swedish consultancy company specialized in streaming, VR and gaming.

Show your support

Clapping shows how much you appreciated Eyevinn Technology’s story.