Making magic moments with picture in picture

PIP, PIP, Hooray

Picture in Picture (PIP) is an interesting feature that gives the user a magical experience. Pip is a special case of the multi-window UI. Video playback and camera activities can often be enhanced by using PIP. For example, while watching YouTube you can select another video to add to your playlist while continuing to watch the current video because the running video goes into PIP mode. This is a magical moment that lets you comfortably do two things at once.

Switching with PIP to check your calendar

Another great use of PIP is checking your calendar while you are on a video chat. The video can shrink and move out of the way allowing you to check your appointments. You can stay engaged in the conversation while completing another task.

Using PIP in your app

Starting with Android 8.0 (API level 26), you can easily put your activity into PIP mode.

Your activity must declare that it supports PIP. In your manifest you should set the supportsPictureInPicture flag to true.

<activity 
android:name=".MediaSessionPlaybackActivity"
...
android:supportsPictureInPicture="true" />

Entering PIP on purpose

There are several appropriate times to enter PIP mode. The simplest way is with a button or menu item. Material design provides standard PIP icons; however a menu option might make more sense for your app.

Picture in Picture icon
Alternative Picture in Picture icon

The icon should indicate what happens when entering PIP. For example, if your PIP window shows in the top right of the screen, use an icon to give the user a subtle hint of what to expect.

PIP as an alternative to switching apps

It is often useful to enter PIP instead of leaving an app, for example when the user presses the home or back button.

PIP might be a better experience than closing your app.

When the user selects back, it may be useful to enter PIP mode rather than closing a running video. For instance, the user might be browsing for more content to view, and could continue to watch the original program in PIP mode. If you do enter PIP mode in this way, be sure to provide an option to dismiss the PIP window.

If the user selects home, recent apps, or opens a notification while handling a video call or watching a video, it might make more sense to enter PIP rather than leave the app.

The Android Activity lifecycle has a callback hook that you can use to detect when the user is leaving the app. You can override onUserLeaveHint(), as explained below, and enter PIP mode instead of leaving.

Diving Deeper

When you are in PIP mode, you should hide everything except the most important thing. In most cases, that will be the video or the camera.

Before entering PIP, you should prepare by performing these tasks:

  1. Hide any controls or overlays on the view. For example, hide video playback controls or a camera options menu.
  2. Create a PictureInPictureParams.Builder
  3. Set the aspect ratio based on your view. For example, the Google Maps navigation has a vertical PIP window while YouTube videos appear in a horizontal window.
  4. Optionally add custom actions. For example, provide rewind/fast forward for video playback or a mute action for video calls.
  5. Optionally set a Rect bounds to smooth the transition for the activity into PIP.

Here is an example of entering PIP mode in response to a button:

void minimize() {
if (mMovieView == null) {
return;
}
// Hide the controls in picture-in-picture mode.
mMovieView.hideControls();
// Calculate the aspect ratio of the PIP screen.
Rational aspectRatio = new Rational(
mMovieView.getWidth(), mMovieView.getHeight());
PictureInPictureParams params = mPictureInPictureParamsBuilder
.setAspectRatio(aspectRatio)
.build();
enterPictureInPictureMode(params);
}

Here is an example of onUserLeaveHint() to catch the app switching and enter PIP as a side effect, with no user action required.

public class MyPictureInPictureActivity extends AppCompatActivity {
...
    @Override
protected void onUserLeaveHint() {
super.onUserLeaveHint();
PictureInPictureParams params =
new PictureInPictureParams.Builder()
// Set actions or aspect ratio.
.build();
enterPictureInPictureMode(params);
}
...
}

You should create a PictureInPictureParams to pass to enterPictureInPictureMode(). There is an overloaded method with no parameters that is deprecated. Android Studio can help make sure that you are not using the deprecated API.

Switching in and out of Picture in Picture mode

Entering PIP mode is pretty easy but what if the user restores your window to full-screen? You should restore your activity to how it was before entering PIP. For example, showing the playback controls that you hid. Activity has a callback, onPictureInPictureModeChanged(), that you can override that listens for such events. There is also a callback for Fragments.

If the user restores the window to fullscreen, isInPictureInPictureMode will be false. You want to restore your view to before you entered PIP mode. For example, show the video playback controls or restore the camera’s option menu overlay.

@Override
public void onPictureInPictureModeChanged (
boolean isInPictureInPictureMode, Configuration newConfig) {
    if (isInPictureInPictureMode) {
// Hide the full-screen UI (controls, etc.) while in
// picture-in-picture mode.
...
    } else {
// Restore the full-screen UI.
...
}
}

Note that you get a reference to the new configuration, like you do when the configuration itself changes.

Adding custom actions

Even though you should simplify the UI when in PIP mode, you can still allow user interaction with remote actions. The PictureInPictureParams.Builder has setActions() which takes a list of RemoteActions. If there are more than getMaxNumPictureInPictureActions() actions, the input list will be truncated to that number.

You can add custom simplistic actions that preserve some of your full-screen features, such as a play/pause toggle. The sample app, defines two custom actions; a play/pause toggle and request info.

Custom PIP actions

If you are using MediaSession for video playback, the framework automatically recognizes the session and adds the actions to play/pause, skip next, and skip previous. The actions are handled by the corresponding methods in MediaSession.Callback.

Default actions with MediaSession

The actions are automatically enabled and disabled depending on the MediaSession state. In the picture below, the MediaSession state is set to PlaybackStateCompat.ACTION_PLAY_PAUSE | PlaybackStateCompat.ACTION_SKIP_TO_NEXT

Actions update based on MediaSession state

If you do not need any or all of the actions supplied by MediaSession, call setActions() to specify only the actions you want.

In Summary

Deciding when to enter PIP is the hardest part. PIP can make your app feel magical by deducing user intention or it can make your app feel clunky and annoying. Once you understand your the user’s intent, then the code will fall into place.

The most common flows to enter PIP mode will be as follows:

  1. Trigger from a button
  • onClicked(View), onOptionsItemSelected(MenuItem), etc.

2. Trigger from intentionally leaving your app

  • onUserLeaveHint()

3. Trigger from discovery

  • onBackPressed()

Continue Learning

Check out the following articles and training documents to continue learning about PIP.

To play with PIP, download the sample.

To dive deeper into PIP, read my second article about Navigation with PIP.

We would love to hear your feedback! If you would like to continue the discussion, leave a response or talk to me on Twitter.