Codec wars — Apple drops the hammer on VP9, chooses H.265 and takes a decision for the industry

It seems funny, but this year’s Apple’s WWDC was one of the most import yet, but not due to something said during the 2017’s Keynote. Instead, this low profile presentation will have far reaching implications over the next 8 to 10 years on the video delivery industry, from Google to Netflix. As Apple decided to use H.265 and WebVTT, the whole industry will be forced to follow suit.

The story started back in May 2010 when Google acquired On2 technologies, and specifically all the related IP to the VP8 video codec. Google was not happy with having to rely on an open source but patented video codecs for it’s future plans, and thought that VP8 would be a solution for that problem. VP8 was still a patented technology, although Google vowed never to use those patents as a weapon against users of VP8 encoders or decoders. However, Google actually tried to force it on the industry, announcing that it would support only VP8 on Google Chrome. Fortunately for everyone, this threat never materialised. This was due to a number of factors such as:

  • VP8’s lack of performance against H.264
  • Virtually non existing hardware decoding support

Eventually, Google caved in, and continued to support H.264, which is the de facto standard codec on both web and broadcast world. In the meantime, a number of codecs were rendered obsolete including VC-1, VP8 and MPEG-2.

Fast forward to 2017

Challenges om 2017 are fairly different. Now the industry’s buzzwords are UltraHD, Wide Colour Gamut, HDR and HFR, which means a new codec is needed to handle them all. This time, two alternatives came into consideration: MPEG Group H.265 (AKA HEVC) and Google’s VP9, both corresponding to evolutions to the corresponding previous version (H.264 and VP8, correspondingly).

So, a new codec war was envisioned. However, this time players moved significantly faster and before. Most silicon manufacturers were quite quick to react:

  • Broadcom deployed H.265 supporting chipset to STBs in 2015 (did one of those myself in 2015 :) )
  • Marvel’s Armada 1500, Rockchip’s RK3288 and AmLogic’s S812, all of those announced between 2015 and 2016.

Most of those offerings did not supported VP9 at time of launch, and then, most won’t support it though a microcode update. This creates a significant legacy installed base, for which VP9 is not an option.

On the mobile side, things weren’t that different. Qualcomm announced the Snapdragon 805 processor back in 2014, supporting 4K H.265 decoding, and encoding shortly afterwards. Even today, Qualcomm only supports VP9 decoding and encoding on the flagship Snapdragon 835 processor, leaving all other processors with software decoding as an option. On the encoding side, there’s not option at all. All current smartphones are limited to 4K@30fps as defined on the H.264 L5.1 profile, and even then, at a very high bandwidth, in excess of 50 mbps.

On the Samsung side, things aren’t that different: all Exynos processors since Exynos 7 support H.265 decoding, and only the latest chip, the Exynos 9 supports VP9.

The reason why VP9 hardware support is lacking is fairly easy to explain, and can be demonstrated by this screenshot coming from the VP9 specification’s page:

VP9 specification status on March 2017
VP9 specification status on March 2017

Contrary to H.265, whose format was finalised back in 2013, which first implementations in 2015, VP9 bitstream is still not final, making silicon vendors to take educated guests in order to support it on hardware, including using programmable components, making such silicon more expensive that the H.265 counterparts.

As a side effect of being ready sooner, H.265 now supports more features, such as native 3D support and scalable versions, allowing currently deployed hardware to be compatible with the base streams even if the new advanced features aren’t supported. Why is this important ? Bandwidth. Let’s take as an example a Satellite broadcaster such as Dish (yeah, I know satellite broadcasters are a dying breed, but they still account for more subscribers than IPTV…). It will be possible for such broadcasters to reuse the already broadcasting HD or 4K stream, and add an additional stream for the multi view 3D format, thus saving a significant amount of bandwidth (around 30% from the 30–40Mbps of an 3D 4K channel) and at the same time, being compatible with the already deployed set top boxes.

Where we stand today

The state of the 4K codec war was as follows on January 2017:

Platform H.265 VP9 Web / Browser

IE Yes, using MSE Yes, using MSE Chrome Not Supported. Not clear if can support OS feature. Supported Firefox Not Supported. Not clear if can support OS feature. Supported Safari Supported on MacOS 10.13 Not supported Set Top Boxes Supported and broadly used Most recent Chipsets support. Not usually enabled on software Mobile Android Supported on Snapdragon 805-class onwards (includes Exynos Series 7) Supported on Snapdragon 835-class onwards (Includes Exynos Series 9) iOS Supported on iOS 11 Not Supported

There are 2 opposite perspectives here, and one of them is more important than everything else. To start with, we have the PayTV STBs. Here, you don’t see VP9. However, all 4K content is today based on H.265, no exceptions. there are two main reasons for this:

  • H.265 came faster to the market, even ignoring VP9 spec is still a draft…
  • H.265 offers the best quality for the bit.

What is important on the STB side, is not on the broadcasting world, but on the OTT world instead. As the broadcasters have complete dominium on the STB, it means they decide which codecs to focus on, and pay royalties on. It’s not enough to say that VP9 is “patent free”, which isn’t, or that it is free, but supporting VP9 from both the software and the silicon has a cost, meaning man-hours to create the software drivers and tools to optimize it, and chipset silicon to do the actual work.

On the other hand, for the likes of Netflix and Hulu, what matters most is the best quality of the bit. A better codec means less bandwidth cost, less storage cost, better quality for the same bitrate profiles. Then, neither Netflix nor Hulu have royalties to care about, as those are paid by the STB manufacturer, broadcaster, user, whatever…

The least concerning aspect is the browser. The browser is not really relevant anymore. To start with, it can easily adapt. Given proper hardware support, it’s easy to take advantage of the OS drivers tapping the hardware to decode H.265 or VP9. On the other hand, not having hardware support makes it irrelevant, as neither codec is really effective if decided on software. At the end, we need to look at the big picture, content is not not consumed on the browser any more, with the sole exception of Youtube.

The most import aspect is finally, the smartphone, which is the game changer. Other than PayTV content, which is mostly consumed on the STB, content is consumed on mobile devices, smartphones and tablets. And here is where definitely codecs are decided. The fact of the matter is that Apple decided to support only H.265 for 4K content even for devices which are a few years old. This decision alone creates a huge and healthy user base supporting H.265, and H.265 alone. On the other hand, we have Android devices, and here we have the same old story: Google added VP9 support within Android itself, which means that from new and high end devices, to old or low end devices, all support VP9, even if you consider a slideshow as video. However, most current high end devices, the ones which are actually able to decode H.265 or VP9 either on software or hardware, do already support then both on hardware. This however doesn’t mean that things are equal from H.265 and VP9. Not so high end devices, or mid of the market devices, using Snapdragon 810-class or Exynos-7 CPUs.

The inevitable conclusion is that there is only one 4K codec which most mobile devices will ever support: H.265, and there’s little Google can do to prevent this, except actually forbid Android manufacturers from supporting it…

Added the mobile market with the STB market, which is already biased for H.265 means that the industry will inexorably elect H.265 as the next generation codec, rendering VP9 to a market niche. From this day on, H.265 will be the codec to choose. Don’t bother with any other until H.267 comes…

Originally published at Too many Bits, too little Bytes.