Playing video by ExoPlayer

Hello everybody, I write this blog for speak in Android Bangkok (or Droidcon) 2018 at 31 March 2018 and my topic is about ExoPlayer which we use this in Fungjai, music streaming application, and new product application on Android.

Do you know which playback library that YouTube use in Android?

This is BNK48 lasted song and show about portrait and landscape

I divide 3 main topic in this session; Introduction, How to use?, and Additional.

Are you ready to adventure this?

Introduction

Before you create the media application with ExoPlayer, you need familiarize it.

What’s ExoPlayer?

ExoPlayer is an open source media playback library for Android
by Google which write in JAVA and have more advantages than MediaPlayer such as minimal, flexible, and stable.

Exoplayer features are play video and audio, shuffle, repeat, subtitle, playlist, caching/downloading, playing ads, live streaming, album art, offline, cast extension and more.

Recap from Google I/O 2017

I recapped important informations from google I/O 17 at 17 May 2017 which speakers are Oliver Woodman and Andrew Lewis.

ref: https://www.youtube.com/watch?v=jAZn-J1I8Eg

Release Timeline

I check for important version of ExoPlayer from Google I/O 17 when in version 2.x to now.

• r1.2.3 (25 Mar. 15) : This is first version release to developer
• r2.0.0 (14 Sep. 16) : This is first version of Exoplayer 2.x which major iteration of the library
• 2.6.0 (23 Nov. 17) : This is first version which not have r in front of version number
• 2.7.0 (22 Feb. 18) : This release have new features and bugs fixed ex. Player Interface, UI Component, Buffering, Cast Extension, Caching, and more in release note.
• 2.7.2 (29 Mar. 18) : This is lasted ExoPlayer version and fixed for some minor bugs

for more information about release date

exoplayer

and release note

google/ExoPlayer

This is overview for core feature and supporting file for streaming format, audio & video, subtitle, metadata, extension.

ref: https://www.youtube.com/watch?v=jAZn-J1I8Eg

Compare MediaPlayer vs ExoPlayer

ref: https://www.youtube.com/watch?v=jAZn-J1I8Eg

• MediaPlayer support API level 1 to current Android version and ExoPlayer is required minimum API level 16.

As you seen, the minimum android version in your current project is API level 16 which 99.2% of active android devices.

I check this from Android Studio

• MediaPlayer is not support advanced use case. For example, adaptive playback (for support streaming formation such as smooth streaming, DASH, and HLS), media composition, caching, and more.
• MediaPlayer is black box then cannot get control over the inner working in player but ExoPlayer is designed really to be very customizable and extensible.

This is diagram for compare about where player actually lives when using MediaPlayer and ExoPlayer. The MediaPlayer implementation is actually in the Android operating system and above in your application and in contrast you use ExoPlayer in your application for custom something to you use.

ref: https://www.youtube.com/watch?v=jAZn-J1I8Eg

Which applications are using ExoPlayer?

Google developeded ExoPlayer for use in YouTube which huge video streaming service, Google Play Movie, Google Photos, Youtube gaming, Google Play Newsstand before release it to developer.

ref: https://www.youtube.com/watch?v=jAZn-J1I8Eg

Over 140,000 applications in Google Play Store are using ExoPlayer for play media in these application such as Vevo, Twitter, BBC iPlayer, Netflix, Spotify, Facebook, Whatsapp, Twitch, and more applications include Fungjai.

ref: https://www.youtube.com/watch?v=jAZn-J1I8Eg and add this text in my slide

How to use?

ExoPlayer is required Android 4.1 (API level 16). But some features are required higher version.

Configuration

First, add the ExoPlayer dependency in build.gradle in your module which current version is 2.7.2

implementation 'com.google.android.exoplayer:exoplayer:2.7.2'

Then add internet permission in your manifest file to get video url, read/write storage are optional for caching, play media in devices or something like that.

<uses-permission android:name=”android.permission.INTERNET”/> 
<uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />
<uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE" />

Sample use

You can add ExoPlayer in your layout file. That sound easy!

<!-- activity_player.xml-->
<com.google.android.exoplayer2.ui.SimpleExoPlayerView
   android:id="@+id/video_view"
   android:layout_width="match_parent"
   android:layout_height="match_parent"/>

Add function for initialize player and release player.

• Add initializePlayer() function to create new Exoplayer in player view. I set my player to play when it ready and seek to current windows. If player already created, prepare media source from url.
• Add releasePlayer() function to get playback position, current window value, play or pause state, release player and set the player to null.

How to use ExoPlayer in Activity/Fragment?

• use initializePlayer() at onStart() and onResume() for init player
• use releasePlayer() at onPause() and onStop() for release player before destroy activity or fragment

Playback States

For about playback states, have 4 states in player

ref: http://google.github.io/ExoPlayer/doc/reference-v1/com/google/android/exoplayer/ExoPlayer.html

1. Idle (Player.STATE_IDLE) : nothing to play media
2. Buffering (Player.STATE.BUFFERING) : more data needs to load
3. Ready (Player.STATE_READY) : can start playback
4. Ended (Player.STATE_END) : playback ended

If you handle about this, you get playback state at onPlayerStateChanged() when implementation by Player.EventListener.

private String status;
@Override
public void onPlayerStateChanged(boolean playWhenReady, int playbackState) {
    switch (playbackState) {
        
        case Player.STATE_BUFFERING:
            status = PlaybackStatus.LOADING;
            break;
        case Player.STATE_ENDED:
            status = PlaybackStatus.STOPPED;
            break;
        case Player.STATE_IDLE:
            status = PlaybackStatus.IDLE;
            break;
        case Player.STATE_READY:
            status = playWhenReady ? PlaybackStatus.PLAYING : PlaybackStatus.PAUSED;
            break;
        default:
            status = PlaybackStatus.IDLE;
            break;
    }
}

Sample output for this code is application for streaming audio or video and has playback in player.

but you forgot something, try think again for initialize and release player, what’s missing?

Supported formats

In initializePlayer(), your player must have MediaSource for prepare media from url and for supported formats have difference MediaSource.Factory

• The format of regular media such as mp3 and mp4 :

return ExtractorMediaSource.Factory(
new DefaultHttpDataSourceFactory(userAgent)).createMediaSource(uri)

• Playlist format in internet radio or music streaming such as m3u8 :

return HlsMediaSource.Factory(
DefaultHttpDataSourceFactory(userAgent)).createMediaSource(uri)

• Adaptive streaming technologies such as DASH, SmoothStreaming and HLS

val dashChunkSourceFactory = DefaultDashChunkSource.Factory(DefaultHttpDataSourceFactory("ua", BANDWIDTH_METER))
val manifestDataSourceFactory = DefaultHttpDataSourceFactory(userAgent)
return DashMediaSource.Factory(
dashChunkSourceFactory, manifestDataSourceFactory).createMediaSource(uri)

Bonus : You can play audio/video in Exoplayer from file in your device.

val bandwidthMeter = DefaultBandwidthMeter()
val videoTrackSelectionFactory = AdaptiveTrackSelection.Factory(bandwidthMeter)
val trackSelector = DefaultTrackSelector(videoTrackSelectionFactory)

val defaultBandwidthMeter = DefaultBandwidthMeter()
val dataSourceFactory = DefaultDataSourceFactory(
        context, Util.getUserAgent(context, "SongShakes"), defaultBandwidthMeter)
val videoSource = ExtractorMediaSource.Factory(dataSourceFactory).
        createMediaSource(Uri.fromFile(File(filename)))

Manage for orientation and no lock screen

You can watch video in full screen by rotate screen to landscape but have problem about play video again at begin.

What’s solution to solve this problem?

First solution in my thinking is Android Architecture Component but this is so difficult for this.

But I found best solution for me is no complex and original solution to solve this. This solution is add config change for player activity at manifest file.

<activity android:name=".PlayerActivity"
    android:configChanges="keyboard|keyboardHidden|orientation|screenSize|screenLayout|smallestScreenSize|uiMode">
</activity>

BONUS : If you want play video fullscreen with landscape only, you add this for check configuration change with rotate device.

The result for add config change is manifest file is continuous playing video when user rotate screen.

and you can add android:keepScreenOn=“true” in parent layout in activity_player.xml to avoid screen locking while video playing.

<LinearLayout     xmlns:android="http://schemas.android.com/apk/res/android"
xmlns:app="http://schemas.android.com/apk/res-auto"android:layout_width="match_parent"
android:layout_height="match_parent"
android:keepScreenOn="true">
...
</LinearLayout>

Custom controller of player

You can custom user interface for player controller and custom some function in your player.

For custom some function in player : You can add behaviour attribute of your playback. I think every developer don’t use all for set your playback behaviour, or not?

Then I show you a sample custom playback for useful to used in you project.

<com.google.android.exoplayer2.ui.SimpleExoPlayerView
   android:id="@+id/video_view"
   android:layout_width="match_parent"
   android:layout_height="match_parent"
   app:use_controller="false"
   app:show_timeout="10000"
   app:repeat_toggle_modes="one"
   app:fastforward_increment="30000"
   app:rewind_increment="30000"/>

• app:use_controller : “true” for show controller and “false” for hide it.
• app:show_timeout : The time of control is hidden after the user used it. This sample hide controller playback after 10000 ms (10 seconds)
• app:repeat_toggle_modes : “none” for no repeat, “one” for play repeat 1 audio or video, “all” for repeat playlist
• app:fastforward_incerment : The time of fast forward after you click fast forward button. The sample is fast forward 30000 ms (30 seconds)
• app:rewind_increment : The time of rewind after you click rewind button. The sample is rewind 30000 ms (30 seconds)
• app:resize_modes : set for content in player size, default is fit with native aspect ratio and you can resize to fill for fit player into layout

For custom player controller :

At first, you create exo_playback_control_view.xml for custom playback style with override layout.

I have 2 sample playback view to explain about custom style

Example 1 :

• playback have play/pause, fast forward, and rewind button only
• add repeat button

I show summary layout of new playback exo_playback_control_view.xml which have 2 paths are playback button and timebar

You can custom 7+1 buttons in playback

• 7 default button : previous, rewind, shuffle, play, pause, forward, next
• 1 special button : repeat

For default button, so easy to add each button in playback layout file by add ImageButton, and custom button colour with tint, button drawable with style from ExoPlayer

For repeat button, I found repeat button id but not found repeat button style then I set style for repeat button to @Style/ExoMediaButton

and add app:repeat_toggle_modes=”one” in player_activity.xml to use repeat button in your playback

Full layout file exo_playback_control_view.xml for example 1

<?xml version="1.0" encoding="utf-8"?>
<LinearLayout xmlns:android="http://schemas.android.com/apk/res/android"
    android:layout_width="match_parent"
    android:layout_height="wrap_content"
    xmlns:app="http://schemas.android.com/apk/res-auto"
    android:layout_gravity="bottom"
    android:layoutDirection="ltr"
    android:background="#11000000"
    android:orientation="vertical">

    <LinearLayout
        android:layout_width="match_parent"
        android:layout_height="wrap_content"
        android:gravity="center"
        android:paddingTop="4dp"
        android:orientation="horizontal">

        <ImageButton android:id="@id/exo_rew"
            android:tint="#FF5D29C1"
            style="@style/ExoMediaButton.Rewind"/>

        <ImageButton android:id="@id/exo_repeat_toggle"
            android:tint="#FF5D29C1"
            style="@style/ExoMediaButton"/>

        <ImageButton android:id="@id/exo_play"
            android:tint="#FF5D29C1"
            style="@style/ExoMediaButton.Play"/>

        <ImageButton android:id="@id/exo_pause"
            android:tint="#FF5D29C1"
            style="@style/ExoMediaButton.Pause"/>

        <ImageButton android:id="@id/exo_ffwd"
            android:tint="#FF5D29C1"
            style="@style/ExoMediaButton.FastForward"/>

    </LinearLayout>

    <LinearLayout
        android:layout_width="match_parent"
        android:layout_height="wrap_content"
        android:layout_marginTop="4dp"
        android:gravity="center_vertical"
        android:orientation="horizontal">

        <TextView android:id="@id/exo_position"
            android:layout_width="wrap_content"
            android:layout_height="wrap_content"
            android:textSize="14sp"
            android:textStyle="bold"
            android:paddingLeft="4dp"
            android:paddingRight="4dp"
            android:includeFontPadding="false"
            android:textColor="#FFFFFFFF"/>

        <com.google.android.exoplayer2.ui.DefaultTimeBar
            android:id="@id/exo_progress"
            android:layout_width="0dp"
            android:layout_weight="1"
            android:layout_height="16dp"/>

        <TextView android:id="@id/exo_duration"
            android:layout_width="wrap_content"
            android:layout_height="wrap_content"
            android:textSize="14sp"
            android:textStyle="bold"
            android:paddingLeft="4dp"
            android:paddingRight="4dp"
            android:includeFontPadding="false"
            android:textColor="#FFFFFFFF"/>

    </LinearLayout>

</LinearLayout>

Example 2 : custom play/pause button and timebar which design by UI designer

I divide this player to 2 paths

(1) Playback Button : You formally set ImageButton with size and drawable with custom.

<!-- Before -->
<ImageButton android:id="@id/exo_play"
   android:tint="#FF5D29C1"
   style="@style/ExoMediaButton.Play"/>
<!-- After -->
<ImageButton android:id="@id/exo_play"
   android:layout_height="50dp"
   android:layout_width="50dp"
   android:background="@drawable/circle_purple_transparent"
   android:src="@drawable/ic_play_player"/>

(2) TimeBar and Duration:

For duration, you can edit same with normal TextView.

For timebar, you can change timebar size and color at scrubber button, played, unplay, and buffer in timebar.

Bonus : Some applications use unplayed_color and buffered_color in same color. I think for any color in application which approve from UI/UX designer and brand CI (Color Index not Continuous Integration in this context).

Full layout file exo_playback_control_view.xml for example 2

<?xml version="1.0" encoding="utf-8"?>
<LinearLayout xmlns:android="http://schemas.android.com/apk/res/android"
    android:layout_width="match_parent"
    android:layout_height="wrap_content"
    xmlns:app="http://schemas.android.com/apk/res-auto"
    android:layout_gravity="bottom"
    android:layoutDirection="ltr"
    android:background="#11000000"
    android:orientation="vertical">

    <LinearLayout
        android:layout_width="match_parent"
        android:layout_height="wrap_content"
        android:gravity="center"
        android:paddingTop="4dp"
        android:orientation="horizontal">

        <ImageButton android:id="@id/exo_play"
            android:layout_height="50dp"
            android:layout_width="50dp"
            android:background="@drawable/circle_purple_transparent"
            android:src="@drawable/ic_play_player"/>

        <ImageButton android:id="@id/exo_pause"
            android:layout_height="50dp"
            android:layout_width="50dp"
            android:background="@drawable/circle_purple_transparent"
            android:src="@drawable/ic_pause_player"/>

    </LinearLayout>

    <LinearLayout
        android:layout_width="match_parent"
        android:layout_height="wrap_content"
        android:layout_marginTop="4dp"
        android:gravity="center_vertical"
        android:orientation="horizontal">

        <TextView android:id="@id/exo_position"
            android:layout_width="wrap_content"
            android:layout_height="wrap_content"
            android:textSize="14sp"
            android:textStyle="bold"
            android:paddingLeft="4dp"
            android:paddingRight="4dp"
            android:includeFontPadding="false"
            android:textColor="#FFFFFFFF"/>

        <com.google.android.exoplayer2.ui.DefaultTimeBar
            android:id="@id/exo_progress"
            android:layout_width="0dp"
            android:layout_weight="1"
            android:layout_height="16dp"
            app:bar_height="1dp"
            app:played_color="@color/colorAccent"
            app:unplayed_color="#FFFFFFFF"
            app:buffered_color="#FFFFFFFF"
            app:scrubber_color="#FF5D29C1"/>

        <TextView android:id="@id/exo_duration"
            android:layout_width="wrap_content"
            android:layout_height="wrap_content"
            android:textSize="14sp"
            android:textStyle="bold"
            android:paddingLeft="4dp"
            android:paddingRight="4dp"
            android:includeFontPadding="false"
            android:textColor="#FFFFFFFF"/>

    </LinearLayout>

</LinearLayout>

PS. you can create new playback control view in new file and add this in your Exoplayer by put your playback layout file in app:controller_layout_id.

<com.google.android.exoplayer2.ui.SimpleExoPlayerView
    android:id="@+id/player_view"
    android:layout_width="match_parent"
    android:layout_height="match_parent"
    app:use_controller="true"
    app:player_layout_id="@layout/exo_simple_player_view"
    app:controller_layout_id="@layout/exo_playback_control_view"/>

ฺBonus : If you handle play/pause without playback, you can use this in your code. Use player.playWhenReady = false for pause media playing and player.playWhenReady = true for play media and save state with playbackState.

fun pausePlayer() {
    player.playWhenReady = false
    player.playbackState
}
fun startPlayer() {
    player.playWhenReady = true
    player.playbackState
}

ref:

How to pause ExoPlayer 2 playback and resume (PlayerControl was removed)

Chunk List

ExoPlayer support chunk list for loading stream media then I talk about chunk working but I don’t talk about chunk in ExoPlayer library.

This is example for chunk in Fungjai website (https://www.fungjai.com) which show chunk list for play 1 song.

Example, this is a audio media in streaming and server divide path of media to download buffer after position of scrubber, that called it “Chunk”.

Then you seek scrubber to nearly half of song, server check buffer.

If not have chunk list near this, server download chunk buffer to cover scrubber position and continue playing music.

For chunk size, system divide balance size like 10 second per each chunk. You see chunk size is 6 second, don’t worry about this because this is end of chunk list in streaming media.

Add player service

Do you have some question in your mind?

• User want play background after lock screen or on in other application
• Have playback notification for this
• something like that, blah blah…

Your application need a service for something like that at above.

These are 3 types of Service are Background Service (not work directly with user), Foreground Service (show service to user and must display a status bar icon), and Bound Service (offers a client-server interface)

This application use bound service because conversion with component and service all time and working with queue.

This is example application that use bound service for play music in background.

Overview Exoplayer working with service

• PlayerNotificationManager is class for manage about notification playback in bound service.
• PlayerService handle player service working
• PlayerManager manage about service and called by Activity/Fragment which use service such as PlayerFragment and BaseFragment class

At First you create PlayerService.java class for your service in application

public class PlayerService extends Service implements AudioManager.OnAudioFocusChangeListener, Player.EventListener {
...
}

and add this in manifest file.

<service android:name=".player.PlayerService"/>

I create service class follow service bound lifecycle at right of this picture include

ref : https://developer.android.com/guide/components/services.html

• onCreate() : setup important use in service such as media notification manager, media session, and Exoplayer

• onBind() : bind service by component with called bindService() and send data to use by Intent.

In this case, return service in this class.

protected class PlayerBinder extends Binder {
    PlayerService getService() {
        return PlayerService.this;
    }
}
private final IBinder playerBind = new PlayerBinder();
@Nullable
@Override
public IBinder onBind(Intent intent) {
    return playerBind;
}

• onUnbind() : unbind service by component with called unbindService() and sent boolean for rebind service.

In this case, if playback status is idle, player service is stoped.

@Override
    public boolean onUnbind(Intent intent) {
        if (status.equals(PlaybackStatus.IDLE))
            stopSelf();

        return super.onUnbind(intent);
    }

• onDestroy() : reset value and return resource to system

In this case, release and remove listener in player, cancel notify notification playback, and release media session.

@Override
public void onDestroy() {
    pause();

    exoPlayer.release();
    exoPlayer.removeListener(this);

    notificationManager.cancelNotify();

    mediaSession.release();

    super.onDestroy();
}

How to use service?

• You can call service.bind() at place for init service and call service.unbind() for unbind service.

class PlayerFragment : Fragment() {
    lateinit var playerManager: PlayerManager
    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)
        playerManager = PlayerManager.with(context)
        playerManager.bind()
        ...
    }
    ...
    override fun onStop() {
        super.onStop()
        playerManager.unbind()
    }
}

I think very complex for explain about service class in my project, then I tour my project code for this.

Then I bring you to tour my project.

but I cannot bring you tour my code in this blog, then you can watch it in this link.

mikkipastel/VideoPlanet

Additional : For more information about this

I collect for some interesting about ExoPlayer for my work and tell the source for you to study it.

Official ExoPlayer Resource

• This is ExoPlayer github

google/ExoPlayer

and guideline for this

ExoPlayer - Developer guide

• This is ExoPlayer developer blog which you can get lasted news for new feature or sample from them.

google-exoplayer

• You can do this codelab for understanding this.

Media streaming with ExoPlayer

Interesting libraries

• This library is custom ExoPlayer to used it easier and it look like VideoView.

brianwernick/ExoMedia

PS. trim video library is

titansgroup/k4l-video-trimmer

• This library is add filtering video in ExoPlayer

MasayukiSuda/ExoPlayerFilter

and can save video with filter to new video file.

MasayukiSuda/Mp4Composer-android

But if you save filter and watermark, you must save 2 round for this because this library see watermark is a filter then take twice as long.

• For Caching in ExoPlayer, Some developer have problem in more issue about this because not clear for how to use.

Developer in this issue recommend this library to easy caching file and I try this but have problem about save full video caching file in storage.

reference from https://github.com/google/ExoPlayer/issues/420#issuecomment-204251112

danikula/AndroidVideoCache

Exoplayer developer add blog tutorial for caching in medium

Pre-caching/Downloading progressive streams in ExoPlayer

• Interesting topic for me is cast extension, it cast into your app to some device such as smart TV.

New Cast extension (and demo app)

Finally #Advertising

Fungjai is a music streaming application by Thai Developer, Thai company. You can listen music from indies music from Thai and Asian Artist (Japan, Taiwan, Indonesia) and have some feature in this.

• Discover : Top 20 charts, Recommend, New Album, and New Artist
• Browse : you can search song from your mood or genre
• Playlist : playlist by Fungjai staff that have 3 type; Mood & Genre Playlist, Theme Playlist (heartbroken), and Feature Playlist (Recommend song, re-live, DJ).
• My Music : Your favorite (music, playlist, album) and your recently played

This application use ExoPlayer for music streaming player and add service for background listen music.

Fungjai — Thai Music — Android Apps on Google Play

P.S. like or follow my page for lasted blog and content from me ><

and this is my blog :D

MikkiPastel Studio