ExoPlayer from the other side
My name is Ross Beazley and I work for the BBC in the Mobile Platforms team. Mobile Platforms make components and libraries for many of the BBC’s mobile apps including the BBC’s Standard Media Player (SMP). SMP is used for the playback of unencrypted BBC content and at its core, on Android, uses ExoPlayer.
So why did we opt to use ExoPlayer? The BBC’s strategic direction for media streaming on TVs, Desktop and Mobile is to move towards the streaming technology DASH (and specifically the DVB DASH, see document A168). ExoPlayer fits in well with this strategy as it supports the sort of DASH streams the BBC produces.
This post is about a few of the things we did to use ExoPlayer (version 1) for some of the BBC’s use cases. As it happens, during the writing of this post the ExoPlayer v2 branch became stable, however reading this may still be of use. If you are interested in reading more about SMP you can do so on the BBC blogs here.
ExoPlayer comes with a large number of useful parts, much like a box of Lego, and it is up to you how you plug all these bits together. But before we get bogged down with detail it’s worth just understanding what the BBC wants from the standard media player.
SMP, for our clients like iPlayer, should:
1. Be trivial to use,
2. Play both on-demand (catch-up) and live content (like BBC1),
3. Seek forwards and backwards through a stream,
4. Transition from embedded to fullscreen,
5. Monitor quality of service,
6. Integrate playback with Chromecast,
and many many other features.
Most of these features require integrations with ExoPlayer and I hope by summarising a couple of these here they may be of use or inspiration to you on your journey.
What follows describes a) how we modelled “media time” in the system (and thus implemented seeking) in a way that unifies live and on-demand streams, and b) managed the async lifecycle of the UI.
Modelling On-demand Media Time
At any point you can query ExoPlayer for the current playback position and duration of the stream. For SMP we wanted to simplify things for our clients (of which our player UI is an example), and so we made time an observable property. This property combined playback position and duration, in essence a MediaPosition coordinate, from ExoPlayer. You may argue that duration was not something that really changed with an on-demand asset and thus is inefficient, but stick with it and I hope all will become clear.
SMP uses a ScheduledExecutorService to poll ExoPlayer for playback position and duration changes. Observers held in a collection are notified of the new values using the single MediaPosition object. We used a CopyOnWriteArrayList to hold observers. The majority of operations on this collection are reads and this Collection type avoids concurrent modification exceptions if we add or remove an observer during event propagation. The frequency of polling was a tune-able SMP parameter and could be overridden at construction time if required.
The MediaPosition could now be bound to our seekbar, allowing the player UI (or any client) to dispatch seeks to SMP. At this point, support for the on-demand use case was complete.
Modelling Live Media Time
Unlike on-demand, which has a duration, live playback has a “rewind window”. This means its default position when playing is the “live edge” — the latest point in the stream, the opposite for on-demand. If you start watching BBC One you start watching what’s on TV right now and have the option to rewind the stream back in time. The BBC only keeps about two hours’ worth of media for its live streams available i.e. a two hour rewind window. ExoPlayer can report the availability of this rewind window and we shall go into this in a little more detail in just a moment. Therefore a live stream has a current position, an end time and a start time and these values in general always increase with the passage of time.
In order to keep things simple SMP would like to present the same MediaPosition object for live playback, as really it is the same observable time related state in the system. The only real difference between on-demand and live is the assumption that on-demand starts at a position of zero and the duration is really the end time of the stream. With a little property renaming to remove this “starts at zero” assumption we can use the same MediaPosition object for both on-demand and Live playbacks. Our MediaPosition object now looks like this:
But how do we create a MediaPosition object for a live stream? We have a little more work to do with ExoPlayer. The current position is still read from the main ExoPlayer instance but to calculate the stream start and end time requires us to use a new DashChunkSource.EventListener attached to our DashChunkSource. This listener will be notified when there is a change in the size of the rewind window giving us the current start and end times (in milliseconds since the epoch). We then calculate the size of the rewind window, and the live edge latency (the difference between the end time of the stream and the current time).
When a “live” MediaProgress object is created we determine the EndTime by subtracting the live edge latency from the current system time. The start time is simply the end time less the rewind window. The current position is simply read from ExoPlayer.
This MediaProgress object can be bound into our seek bar in the same way as for on-demand. This allows us to seek within live and on-demand streams in the same way. So just as James Clerk Maxwell unified electricity and magnetism into one force, we have unified our concept of time into one object — ok so maybe not as grand, very tenuous and certainly less useful.
Now that we have time and seeking sorted; we turn our attention to the UI and displaying video.
Asynchronous UI
SMP, like ExoPlayer, can be either run headless or with our UI on display. To support this we expose a method to attach or detach a surface to SMP. Before I go into how (and why) this implemented I shall explain a) how our default transition to full-screen action is performed and b) how we manage a “video card” list.
Transition to full-screen
When an “embedded” playout view is used in an app (you know, like YouTube when it’s not fullscreen) a surface is created and this is ultimately sent (in a message) to the ExoPlayer video track renderer. When the switch to fullscreen button is pressed we fire up a new fullscreen activity, inject the SMP instance using an Application.ActivityLifecycleCallbacks listener, and create a new playout view for the activity. This again results in a surface being created and sent to the video track renderer. This replaces the previous surface sent to the video track renderer by the embedded view. When the fullscreen activity is closed, its surface is detached from SMP and playback should continue on the “embedded” playout view.
Video Card List
SMP supports the use of multiple views on a single instance. This is how we implement a video card list. A different piece of content is displayed in each view and only one piece of content can play at any time. Pressing play on a card results in a surface being created and attached to SMP. If playback is initiated on another card a surface is created and attached to SMP and the surface in the other card is detached and destroyed. SMP makes no assumptions about what order this may happen in.
Surface Stack
We maintain a stack of surfaces that have been attached to SMP. When a surface is attached to SMP we push it onto the top of the stack and when that UI is removed and surface destroyed it is detached from SMP and is popped off the stack.
The surface at the top of the stack is the one attached to the video track renderer, when the top surface is popped (removed) we attach the new top of the stack to the video track renderer. This might be null of course if the stack is now empty which results in the video decoder getting disabled.
This solution covers both the embedded, fullscreen and video card list use cases. It also allows us to move towards an enhanced secondary screen experience when you plug in an HDMI cable, the TV showing the main video content and controls remain on the main SMP UI.
This article has covered only a small number of the ways we use ExoPlayer and if time permits I will follow up with more details including what we learned on our transition to ExoPlayer 2.x, how we decoupled ourselves from ExoPlayer, how we shaped our acceptance test suite, how we avoided conditionals with a “one class per state” FSM and the benefits (and pitfalls) or raising domain events on a global event bus.
If you are interested in reading more about building mobile components for media playback at the BBC please head over to our technical blogs. If you want to get stuck in to building mobile components for media playback, the BBC needs you — drop me a line.
