Improved decoder reuse in ExoPlayer

Published in

AndroidX Media3

5 min readApr 24, 2019

ExoPlayer 2.10 improves the way decoders are reused. By reusing decoder instances in more cases, these improvements can reduce the time required to switch from playing one piece of media to another. This post describes what’s changed, and what you need to do to make sure that your app benefits from the improvements.

The importance of decoder reuse

By default, ExoPlayer relies on platform decoders to decode video (e.g. H.264) and audio (e.g. AAC) samples during playback. These decoders are accessed via Android’s MediaCodec API.

Instantiating, configuring and starting platform decoders can be slow for a number of reasons, including the time taken by the underlying platform to allocate the decoder’s input and output buffers. In fact, the time required to setup (i.e. instantiate, configure and start) the decoders can be a significant percentage of overall playback startup latency. This is particularly true when playing local content, for which network latency is not a factor. The annotated systrace below shows the start of a local playback on a Galaxy S8, where the media being played contains 1080p H.264 video along with AAC audio. In this particular run, the time taken to setup the decoders accounts for 60% of the total startup latency.

In case you’re curious, loading the start of the video from local storage and parsing the video and audio formats from it account for the majority of the 80ms of latency before codec setup (codec setup cannot begin until this information is available). The 30ms of latency after codec setup is the time taken for the video decoder to decode the first frame.

We cannot avoid setting up decoders, since they’re essential for media playback. However, if we already have decoder instances, reusing them can reduce the startup latency for subsequent playbacks. This can significantly improve the experience in use cases where the user frequently advances to playing another piece of content, such as in Snapchat’s Story UI.

Decoder reuse prior to 2.10

To understand decoder reuse prior to 2.10, it’s necessary to understand the three states of renderer components within an ExoPlayer instance. These are:

The disabled state, in which the renderers do not have streams of media to play and do not hold decoder instances.
The enabled state, in which renderers have streams of media to play and will acquire decoder instances when it’s possible for them to do so (e.g. after reading the stream’s format).
The started state, in which renderers have streams of media and are playing them using the acquired decoders.

Prior to 2.10, ExoPlayer would reuse video decoders when possible for as long as the video renderer remained in the enabled and started states. In particular, this meant that video decoders would be reused when transitioning from one piece of media to the next in a playlist. Audio decoders were not reused across such playlist transitions, and neither video or audio decoders were reused when renderers transitioned through the disabled state. Importantly, re-preparing the player to play a different MediaSource causes this type of state transition, and therefore decoders were not reused in this case.

Improved decoder reuse in 2.10

In ExoPlayer 2.10 we’ve made changes that enable audio decoders to be reused. We’ve also enabled decoder reuse when re-preparing the player to play a different MediaSource, by allowing renderers to hold decoders when transitioning through the disabled state. These changes are summarized in the diagram below.

Enabling audio decoders to be reused reduces the amount of work that the player has to do when transitioning from one piece of media to the next in a playlist, reducing the probability of frame drops when transitions occur.

More significantly, playback startup latency can be reduced if the player is able to reuse decoders when re-preparing the player to play a different MediaSource. The systrace below shows an example where the player is being re-prepared with the same video used in the previous example. By eliminating the need to setup the decoders, startup latency is reduced from 280ms to 110ms.

Requirements for decoder reuse

Decoders cannot always be reused. One obvious requirement is that the decoders held by the player must be the ones that are required to play the new piece of media. Hence it’s not possible to reuse a video decoder when switching between VP9 and H.264 playbacks, for example. The full story is significantly more complicated, with multiple modes of decoder reuse and conditions for each one. These details are beyond the scope of this blog post, however an interested reader can find the relevant code blocks here for audio, and here for video.

Making sure your app benefits from decoder reuse

To ensure that your app benefits from decoder reuse, it’s important that:

For use cases where your app switches between playing a number of videos, use a single ExoPlayer instance for all playbacks, rather than instantiating a new instance each time. This is because decoders are only reused within an ExoPlayer instance, not between them.
When re-preparing the player with a different video, call prepare() with the new MediaSource without calling stop() first. This is because by default, ExoPlayer will still release decoders when stop() is called.
If you need to call stop() but still want decoders to be kept around for potential reuse, enable foreground mode by calling setForegroundMode() on the player. This method is very easy to misuse, so please read the Javadoc carefully and use it only when it’s appropriate to do so!
If you control the content played by your app, make sure that it’s all in the same format (as far as is possible).

Future work

The improvements described above improve performance in cases where decoders can be reused, including re-preparing the player to play a different MediaSource. However, they do not help with the first playback in each user session, for which new decoders always need to be setup. In the future we hope to reduce startup latency for these playbacks too, by exploring options including:

Allowing apps to hint to the player what formats it expects to be played, ahead of the first playback. If the app developer controls the content then it’s usually possible to provide such a hint, and doing so would allow ExoPlayer to setup the necessary decoders ahead of time. This enhancement is tracked by Issue #5795.
Setting up video and audio decoders in parallel, rather than one after the other. This enhancement is tracked by Issue #5796.

This post has provided an overview of the decoder reuse improvements in ExoPlayer 2.10, and what you need to do to make sure that your app benefits from them. As always, please feel free to ask questions and report issues on our issue tracker.