Transforming media with ExoPlayer

ExoPlayer 2.13 includes a new Transformer library module for converting media streams. This post describes what this module does and how to use it.

API capabilities

Unlike the other ExoPlayer libraries, the main purpose of the Transformer API is not media playback. It takes an input media stream, applies changes to it as configured by the app, and produces the corresponding output file.

This API is still at an early stage. New features will be added in future versions and the signature of the existing methods are likely to change as a result.

In the initial version, we have focused on the specific use case of converting slow motion videos to make them suitable for sharing with other apps or uploading to a server. We have also added some other basic functionality. The available transformations are:

• Transmuxing between container formats.
• Track removal.
• Flattening of slow motion videos or, in other words, their conversion into normal videos that retain the desired slow motion effects, but can be played with a player that is not aware of slow motion video formats.

Multiple transformations can be executed sequentially with the same Transformer instance, but concurrent transformations with the same instance are not supported yet.

The input can be a progressive or an adaptive stream, but the output is always a progressive stream. For adaptive inputs, the highest resolution tracks are always selected for the transformation.

Starting a transformation

To transform media, you need to add the corresponding dependency to your app’s build.gradle file.

You can then start a transformation by building a Transformer instance and calling startTransformation() on it. The code sample below starts a transformation that removes the audio track from the input and sets the output container format to WebM:

Other parameters, such as the MediaSourceFactory, can be passed to the builder. startTransformation() receives a MediaItem describing the input, and a path or a ParcelFileDescriptor indicating where the output should be written.

Listening to events

The startTransformation() method is asynchronous. It returns immediately and the app is notified of events via the listener passed to the Transformer builder.

Displaying progress updates

The getProgress() method can be called to query the current progress of a transformation. The returned value indicates the progress state. If the progress state is PROGRESS_STATE_AVAILABLE then the passed ProgressHolder will have been updated with the current progress percentage. The snippet below demonstrates how to periodically query the progress of a transformation:

This can be used to display a progress bar to the user.

Flattening slow motion videos

We define a slow motion video as a media stream whose metadata points to sections of the stream that should be slowed during playback. Flattening is the process of converting a slow motion video to a regular media format (for example MP4) where the slow motion sections are played at the requested speed. The slow motion metadata is removed, and the video and audio streams are modified so as to produce the desired effect when the output is played with a standard player (that is, a player that is not aware of slow motion formats).

To flatten slow motion streams, use the setFlattenForSlowMotion() builder method.

This allows apps to support slow motion videos without having to worry about handling these special formats. All they need to do is to store and play the flattened version of the video instead of the original one.

Currently, only Samsung’s slow motion format is supported. We will likely add other formats in the future.

Inner workings

The Transformer uses a SimpleExoPlayer under the hood to extract the input samples, and a MediaMuxer to write the converted samples to the output. Consequently, some characteristics of the Transformer are currently dictated by the MediaMuxer API. The main examples are:

• The minimum required Android version, which is 4.3.
• The supported output formats.

Future work

This module is still at an early stage of development and we have many ideas to improve it, including:

• Adding other transformations, such as transcoding or resolution change.
• Adding support for playlists.
• Flattening other slow motion formats.

Note that these are just ideas. We haven’t decided yet whether we will implement them.

If you would benefit from a particular feature, feel free to request it on our issue tracker. We are also happy to consider high quality pull requests!

Even though this is only the first version, we think that the Transformer API has great potential and can facilitate the implementation of various use cases. As always, please share your opinion and questions on our issue tracker. Thanks for reading!