Drop-in Android Video — Exoplayer2 with Picture in Picture
Beautiful, interactive video content done easy.
Video has been shown to increase engagement in apps and with content. How else could gif servicing apps be worth tens of millions and Youtube influencers be selling billions of dollars of product?
The problem is that video in Android can be difficult. You are given the assumption that the video types you will support are pretty basic : embedded MPEG video, URLs to MP4/AVI files, and HLS Streaming. A naive approach would be to use the embedded video player sent with Intents, but that takes the user out of your app. You think, 🤔 “ I’ll just use the Android VideoView class with AppCompat .” Then you realize that it is nearly impossible to make the experience consistent across users because of codec support and whatever the phone manufacturer may have customized in the video player. Finally you settle on Exoplayer2 because it is the package everyone, including Google, uses. Upon further inspection, you quickly realize that it requires a lot of code to make it work and the example application on the Github is confusing, far too extensive for what you are doing. Another route you could take is ExoMedia by Brian Wernick, but it trades extensibility, theming, and control for simplicity.
Fortunately, you only need the Exoplayer2 libraries to have a fantastic video player experience within your app.
SimpleExoplayer “2” the Rescue
First you need to make an app which can receive a URL and play it using Exoplayer. For this, you will need SimpleExoplayer and the Playerview from the UI package. Playerview has easy to manipulate styling/theming elements as well as available media controls. SimpleExoPlayer gives you a preconfigured player to work with without having to handle deeper logic yourself.
Start by adding the Gradle dependencies to your build.gradle :
implementation 'com.google.android.exoplayer:exoplayer-core:2.X.X'
implementation 'com.google.android.exoplayer:exoplayer-hls:2.X.X'
implementation 'com.google.android.exoplayer:exoplayer-ui:2.X.X'
implementation 'com.google.android.exoplayer:extension-mediasession:2.X.X'Prepare a layout for your video:
<?xml version="1.0" encoding="utf-8"?>
<android.support.constraint.ConstraintLayout xmlns:android="http://schemas.android.com/apk/res/android"
android:layout_width="match_parent"
android:layout_height="match_parent"
android:background="@android:color/black">
<com.google.android.exoplayer2.ui.PlayerView
android:id="@+id/playerView"
android:layout_width="match_parent"
android:layout_height="match_parent"
android:layout_gravity="center" />
</android.support.constraint.ConstraintLayout>Implement Exoplayer in a VideoActivity (Kotlin) :
Quick Note : Exoplayer should be set up in the onStart() for quick initialization that does not interfere with onCreate().
lateinit var player : SimpleExoPlayer
var mUrl = *some string HLS/Standard video url passed to it*override fun onStart() {
super.onStart()
player = ExoPlayerFactory
.newSimpleInstance(this, DefaultTrackSelector())
playerView.player = player
val dataSourceFactory =
DefaultDataSourceFactory(this,
Util.getUserAgent(this,
applicationInfo.loadLabel(packageManager)
.toString()))
when (Util.inferContentType(Uri.parse(mUrl))) {
C.TYPE_HLS -> {
val mediaSource = HlsMediaSource
.Factory(dataSourceFactory)
.createMediaSource(Uri.parse(mUrl))
player.prepare(mediaSource)
}
C.TYPE_OTHER -> {
val mediaSource = ExtractorMediaSource
.Factory(dataSourceFactory)
.createMediaSource(Uri.parse(mUrl))
player.prepare(mediaSource)
}
else -> {
//This is to catch SmoothStreaming and
//DASH types which we won't support currently, exit
finish()
}
}
player.playWhenReady = true
}override fun onStop() {
super.onStop()
playerView.player = null
player.release()
}
That was almost too easy!
Boost it with Picture-in-Picture
This is where things get interesting.
According to the basic documentation on AndroidDevelopers, this is all you need to get started with Picture in Picture :
//For N devices that support it, not "officially"
@Suppress("DEPRECATION")
fun enterPIPMode(){
if(Build.VERSION.SDK_INT >= Build.VERSION_CODES.N
&& packageManager
.hasSystemFeature(
PackageManager.FEATURE_PICTURE_IN_PICTURE)) {
videoPosition = player.currentPosition
playerView.useController = false
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.O) {
val params = PictureInPictureParams.Builder()
this.enterPictureInPictureMode(params.build())
} else {
this.enterPictureInPictureMode()
}
}Now, I will say that while implementing Picture in Picture support, Google was notified of issues in documentation. It’s better now and includes some of the things like checking if the system has the feature (some devices will not have Picture in Picture because of RAM considerations) which is shown above. 👏
Next, hook up switching to Picture in Picture when the user hits the Recents, Home, and Back buttons here :
override fun onBackPressed(){
if(Build.VERSION.SDK_INT >= Build.VERSION_CODES.N
&& packageManager
.hasSystemFeature(
PackageManager.FEATURE_PICTURE_IN_PICTURE)){
enterPIPMode()
} else {
super.onBackPressed()
}
}/Called when the user touches the Home or Recents button to leave the app.
override fun onUserLeaveHint() {
super.onUserLeaveHint()
enterPIPMode()
}
Google has added some of this to the documentation recently 😀, but not onBackPressed() 🧐 . What could be unique about onBackPressed()?
Permission Handling
The user can turn off this permission as part of the Advanced permissions.
If you are familiar with deeper permission handling as well as the system the normal Permissions tie into, you should just be able to handle this case in AppOpsManager. In that case, the code would look like this :
val appOpsManager =
getSystemService(Context.APP_OPS_SERVICE) as AppOpsManagerif(appOpsManager.checkOpNoThrow(
AppOpManager.OP_PICTURE_IN_PICTURE,
packageManager.getApplicationInfo(packageName,
PackageManager.GET_META_DATA).uid, packageName)
== AppOpsManager.MODE_ALLOWED){
//Picture in Picture is enabled, yay!
}
Oh, there is a problem with that. It’s not valid in Android Studio 😠.
https://github.com/aosp-mirror/platform_frameworks_base/blob/studio-3.1.2/core/java/android/app/AppOpsManager.java#L1640
For the Integer-Op-type, unlike the String-Op-type version, the ability to check the permission is hidden 🙅.
WHY?!?!
One likely answer is that this was a carry-over from N where Google didn’t anticipate Picture in Picture becoming a public feature or Google could be trying to enforce certain developer behavior. Right now, if the user does not have the Picture in Picture mode enabled in these special permissions, it will not work and there isn’t a nice way to check it 😥. For apps, it means that you cannot enable Picture in Picture mode for onBackPressed() without risking needless crashes and an unpleasant user experience.
Pressing the back button seems like the most common use-case for small video content, so this lack of handling the Back button doesn’t entirely make sense.
Again, they could have this fixed in the proper AndroidStudio 3.2.X release, but that isn’t out yet.
So, you need to be a little creative here 🤔 :
fun enterPIPMode(){
if(Build.VERSION.SDK_INT >= Build.VERSION_CODES.N
&& packageManager
.hasSystemFeature(
PackageManager.FEATURE_PICTURE_IN_PICTURE)) { ...
... Handler().postDelayed({checkPIPPermission()}, 30)
}
}@RequiresApi(Build.VERSION_CODES.N)
fun checkPIPPermission(){
isPIPModeeEnabled = isInPictureInPictureMode
if(!isInPictureInPictureMode){
onBackPressed()
}
}
I know, it seems a bit jank. But, 25–30ms to check whether the Activity actually went into Picture in Picture mode is plenty of time, even devices that have very limited memory (512mb + running other apps), to enter Picture in Picture mode as far as the system is concerned. It also gives you enough time to send the onBackPressed() so that the system knows you aren’t holding the user hostage and the delay is neigh imperceptible by the user. Yes, this is under the AOSP “Bad Behavior” timer. 😜
More Picture in Picture oddness discovered
Now that Picture in Picture mode is working and no crashes can really happen, there is another scenario which comes up.
The user hits the “X” or drags the Picture in Picture window down to dismiss. You would expect that this simply calls finish() and the window disappears into the ether. But, it doesn’t. You can still see the activity lingering in the Recents for what seems like forever as you wait for it to be collected………*3 minutes later*………….now. 😞
This is a problem because:
- It doesn’t look good for your app
- It can be a major headache if you want to start up a new Picture in Picture instance because the old one is just lingering there.
If you watch to see what is getting called when the user “closes” a Picture in Picture window you see this being called :
override fun onStop() {
super.onStop()
}Logically, finish() is being called and that is one of the first LifecycleEvents part of that flow. But, not in the traditional sense.
Picture-in-Picture mode is unique, especially in the separate Activity case, because it acts like it is entirely its own app.
To get around this problem and clear out the VideoActivity quickly and efficiently implement this :
override fun onStop() {
....
....
if ((Build.VERSION.SDK_INT >= Build.VERSION_CODES.N)
&& packageManager.hasSystemFeature(
PackageManager.FEATURE_PICTURE_IN_PICTURE)) {
finishAndRemoveTask()
}
}Done. Killed and removed appropriately.
Adding Media Controls to Picture-in-Picture
Start/Stop/Next
By default, Exoplayer + Picture in Picture does not give you the Picture in Picture media controls you see in the Youtube player for instance. Picture in Picture mode depends on a MediaSession to handle that communication. If you have written an AndroidTV application, audioplayer, or video-centric app before, you should be familiar with it. It can also be used for handling media controls from outside devices (like bluetooth headsets), widgets or advanced notifications with media controls. Here, you will be using the MediaSession Extension Library to take care of this. ⏯
Just add this code in the onStart() after handling your Exoplayer :
val mediaSession = MediaSessionCompat(this, packageName)
val mediaSessionConnector = MediaSessionConnector(mediaSession)
mediaSessionConnector.setPlayer(player, null)
mediaSession.isActive = trueOne More Thing
Since Picture in Picture essentially acts like its own Application, it can live on without the Main App.
Yes, if you kill your Main App the Picture in Picture window will live on without you unless you kill it first. That means, if you have connecting information (such as multiple URLs or other streaming info) being sent and resetting the content of the Picture in Picture window that connection will be lost and it will become a ZOMBIE Picture in Picture WINDOW 😱.
What you can do to solve this is register an “Exit Event” Service to capture when the user Force Closes your app. In this Service, you need to override onTaskRemoved(rootIntent:Intent) and send a signal (RxJava Observable/Subscriber, Eventbus, Intent, etc.) which the Picture in Picture’d Activity can catch and call finishAndRemoveTask() to make a clean exit.
Code
There you have it folks! Lots of little gotchas taken care of.
Full code example with a DROP-IN VideoActivity.kt you can use right now.
http://bit.ly/exoplayerPIPexample
It includes some extra, useful features:
- Play/Pause/Resume/SeekTo controls for v ideo
- Show/Hide Exoplayer controls when the user goes in-out of Picture in Picture mode
- Rotation handling (fullscreen and vertical with no placement loss).
