Android and Video: The Good, the Bad and the Ugly
Written by Ian Bird — March 6th 2018
Over the better part of a decade, we’ve continued to develop and improve our Plex for Android app to match the ongoing evolution of the ecosystem. We’ve seen Material Design change the way apps are meant to look and feel, and the Android platform itself introduce new features that enrich the media playback experience (Google Cast is now everywhere!). However, there has been one aspect of Android that we have continued to battle with: video playback! Over the years we’ve come away with many war wounds and stories, but also many victories. In this blog (hopefully the first of many!), we thought we’d reflect on our adventures and give insight into the lengths we’ve gone to slay the beast.
A long time ago (in a galaxy far, far away…) your options for media playback on Android were, let’s say, limited. You would create a VideoView, pass it a URL to the media to play and that was it. Internally, it used the Android MediaPlayer to understand what the linked resource was and how to play it (if it could). Over the years, support for the way in which you could manipulate it was expanded, but an application’s ability to control the experience or capabilities was pretty much non-existent (think of it as a black box!). Applications essentially relied on Google and the device manufacturer to make things work under the hood. However, one frequent criticism of Android has always been the version fragmentation that exists. And the other thing that exists, with any platform… bugs. Whether it was an Android OS issue—which Google would fix but that may or may not be released by a manufacturer—or a manufacturer issue that went unnoticed and only impacted a few small number of applications/scenarios, the experience from a development standpoint was a minefield!
One such example that I had the pleasure of working on was when LG released their G2 mobile. At the time, this device was rapidly becoming one of the most popular devices of our user base. Unfortunately for us though, we had early reports that “no video played”. After a mad rush to find a device, we sat down to investigate the issue. The user reports were right, any video which played *from* Plex Media Server immediately reported failure without any further details (this was going to be fun!). To improve our visibility, we wrote an HTTP proxy which sat between MediaPlayer and the media server, so we could observe exactly what was happening. After much swearing, and maybe some heavy drinking, we stumbled across the fact that the failure would only happen if the HTTP response included one specific header… `X-Plex-Compressed-Content-Length`. This was one of our own (meant to be private!) headers and not something that should cause any side effects. Upon probing a little further, we realised that playback was fixed if we either removed the header from the response, or we mutated it (in the proxy) to be equal to the `Content-Length` value (a common HTTP response header). Our conclusion: the networking implementation used by MediaPlayer on this specific device was using “String.contains” rather than “String.equals” when parsing headers. This is obviously a pretty crazy bug to find in the wild, but one that was the cause of many (totally valid) complaints by our users that their shiny new devices weren’t supported. We productionized our local proxy (turned out to be needed for lots of different device issues!), released it, and #profit.
This is just one example of the lengths to which we went to ensure support over the Android ecosystem. It was expensive (in terms of development effort) and painful! When video playback is a core part of your offering, can you really afford to rely on others to deliver it?
Jumping forward a few years, a number of us were lucky enough to be involved with the release of Android TV at Google I/O 2014. While there, we attended some great developer talks and one such was by Olly Woodman on Google’s ExoPlayer. This open source project was born out of need for Google to provide excellent media support across Android as a whole, a challenge they had been battling themselves with their own applications like YouTube and Play Movies. ExoPlayer was a game changer for us… a feature-rich, extendable, open source player which was decoupled from the Android OS, allowing for fixes and improvements to be controlled independently of OS updates. I’m pretty sure I started integrating it on the flight back to sunny Scotland, and before long we had it up and running. Early versions of ExoPlayer supported a similar set of containers to Android’s MediaPlayer. It was initially focussed on MP4 and MPEG DASH support (although HTTP Live Streaming was later added). Their WebM support was improved to support MKV (WebM is a subset of MKV), which we also contributed to. Before long, we were able to offer a video playback experience that not only matched our previous capabilities, but added new features (we even wrote our own image-based subtitle support — VOBSUB and PGS).
Everything Is AWESOME!!! right? Well, not quite. One of the unique challenges we face is the incredible variety of media our users bring. ExoPlayer is optimised for the common scenarios, but lacks extractor support for files such as AVI, WMV. The containers it does support, and streaming protocols, also only support a subset of codecs. One solution could have been for us to undertake the daunting task of writing our own support for everything that was missing… The alternative? Well, doesn’t Plex’s own transcoder (used in Plex Media Server) already have incredibly rich support for different formats and codecs? Why… yes, it does! Development started nearly a year ago to build on top of Google’s latest version of ExoPlayer (v2) but leverage our own native transcoder to fill in the gaps. This gives us the best of both worlds: a powerful and Android-optimised media player that leverages the extensive support of formats that our transcoder does. If this sounds like an easy task, it wasn’t (but if you think it is, and are a developer, we’d love to meet you!). Our first experiment using this completely new video player stack was our “Local Playback” feature (released in June ‘17). This feature allows any video file located on an Android device, whatever container and codec, to be played back via our application. It demonstrated the power of our new video player, which continues to be built on, with new features constantly being added. We then leveraged the same player stack to bring better Live TV support to both Mobile and TV devices; it even adds software decoding for audio and video which might not be supported by the physical device. Next it was onto Plex News which also leverages our very own video player.
As some people may have seen, we’ve now started using the same (awesome) video player as of Plex for Android v6.13 for library playback too. This means that you can Direct Play an incredibly wide range of media on your Android device, without the need for the server to convert it.
We’d love to hear your feedback on the new player, so if you’re an Android user, come over to our friendly forums and let us know if you run into any issues or just want to let us know how well it’s working!
