Managing Videos on Android

Grey Skold | Android Video Product

Video launched on the Pinterest Android app in 2016 with the goal of bringing a seamless video experience to the app. This included the ability to support multiple videos per screen, while dynamically controlling their playback state by automatically pausing them when they’ve scrolled offscreen, or controlling the number of videos allowed to play simultaneously.

We soon discovered there were many technical challenges we needed to address, such as:

• Managing the playback state of all currently available videos
• Knowing a video’s on-screen visibility percentage
• Providing our developers an easy-to-use video component

Over time we’ve gradually reworked our video architecture to tackle these requirements, and below we’ll dive into how we handled these challenges in our latest video module.

The Video Manager

At a high level, we needed to build a component that would be aware of all video instances (i.e. Views) as well as their related surfaces (i.e. Fragments) available on-screen. Managing the surfaces is critical for monitoring the lifecycle states (i.e. onStart() etc.) that will be applied to the surface’s children, and avoids adding excessive code on the consumer-layer to apply the latest state changes to the Views.

For tracking these key lifecycle events, the Android framework provides us the current state of both the content being shown on-screen, as well as any changes visually affecting our app. The key lifecycle events we listen for are the UI attachment calls (e.g. onAttachedToWindow()) as well as when our host screen changes its display state (e.g. onPause() etc.).

With these callback methods, we attempted to register any videos that have been provided a valid video URL. This will give us our initial list of videos available within our current surface.

In the first iteration of our video framework, we relied on the client code invoking these calls themselves, but as we found this to be unscalable, as it added more complexity when building video features. Instead, we abstracted away the callbacks for registering a video behind the VideoManager by building methods that required the underlying video component to be passed in. From there, the VideoManager would make appropriate calculations behind the scenes. This eliminated the need for consumers to already have predefined knowledge of the video registration process, as it now just worked “out-of-the-box”.

Before

// FooBarFragment.class for FooBar feature
override fun onResume() {
     super.onResume()
     // Required by consumers to implement
     videoView?.apply {
          viewability = Viewability.FullyVisible
          onActivate()
          onViewCompletelyVisible()
     }
}
override fun onPause() {
     // Required by consumers to implement
     videoView?.apply {
          viewability = Viewability.NotVisible
          onDeactivate()
     }
     super.onPause()
}

After

// BaseFragment.class implemented by all screens
override fun onResume() {
     super.onResume()
     videoManager.onResume(this)
}
override fun onPause() {
     videoManager.onPause(this)
     super.onPause()
}
// VideoManager.class internally
override fun onResume(videoSurface: VideoViewSurface) {
     videoSurface.videoViews.forEach {
          registerVideo(it)
     }
}
override fun onPause(videoSurface: VideoViewSurface) {
     videoSurface.videoViews.forEach {
          unregisterVideo(it)
     }
}

Holding onto this list of videos gave us the ability to dynamically set the playback state based on the current visibility of our app. This also provided the flexibility to dynamically change other functionality based on certain metadata properties passed in upon video registration.

For example, we may want all video ads to autoplay, but limit to having only 1 organic video (i.e. Creator generated content) autoplaying on the same surface. By checking the metadata registered on an individual video, we can apply these limitations onto the UI-layer.

We also abstracted away all Pinterest-specific analytics code in order to maintain a clear focus for the Video Manager (managing and playing videos), and to keep the component app-agnostic.

Calculating Viewability

Viewability is defined as the percentage of visible area of a UI component shown on-screen. This measurement is crucial in our understanding of what is currently being displayed to the user. With this information we are able to gather information for our partners regarding the engagement of their content.

In the common case, since the VideoManager holds references to all active videos, we can track the exact coordinate of our views (i.e. getLocationInWindow()) and our device’s screen size in pixels (see DisplayMetrics) to deduce their Viewability on-screen.

We also handle overlapping UI via:

• Providing consumers the option to include a list of “obstruction” Views that could potentially draw over our underlying videos (e.g. toolbars, floating buttons, etc.)
• Callbacks for pop-ups being displayed (i.e. onWindowFocusChanged())
• Components for screen scrolling or UI components going off-screen (see RecyclerView listeners)
• Additional callbacks for when a video surface is shown on-screen (i.e. onResume() etc.).

Building for Developers

While we wanted to reduce the amount of video management complexity exposed to our developers, the largest area of confusion was implementing a new video surface. Therefore,we both abstracted away the complexity for video setup, as well as utilized UI components provided by Google’s PlayerView:

Before

// FooBar video feature, requires custom FooBarVideoView.class of 100+ lines
object : FooBarVideoView(
     context,     // application context
     analytics,   // Analytics object
     url,         // video url
     uid,         // unique ID
     false        // isAd flag
) {
     // configuration flag for custom setup (mute, autoplay, controller, etc.)
     override val videoConfiguration = VideoConfiguration.FOO_BAR
}.apply {
     shouldLoop = true
     videoAspectRatio = aspectRatio
     render(videoMetaData) // loads video, videoMetaData contains: url, isAd, uid
}

After

// Foobar video feature, no custom class required just set flags
PinterestVideoView(context).apply {
     // Optional params for setup/customization
     this.analytics = pinterestAnalytics
     this.mute = false
     this.autoPlay = true
     this.alwaysAutoplay = true
     this.alwaysPlay = true
     this.showMute = true
     this.looping = true
     this.bufferingRule = SHOW_BUFFERING_ALWAYS
}.apply {
     render(videoMetaData) // loads video, videoMetaData contains: url, isAd, uid
}

Another complexity of the video infrastructure was the actual VideoManager architecture itself. In our rewrite, we consolidated a majority of our old components to only the bare bones needed to support a functioning VideoManager

Before

After

Our new VideoManager architecture provides a clear hierarchy of events and components’ relation to one another. This not only looks good on paper, but the refactor alone has removed over ~4,500 lines of code (less than 1/3 the size of the original implementation).

Playing off into the sunset…

Building proper “video management” is a long and arduous process, but over the years we’ve built something that’s truly evolved to help both streamline our development process and Pinner experience. In the future, we hope to open source our work so other developers may contribute to the ongoing effort to handle dynamic video playback. We will continue to iterate on our video client architecture tackling new challenges as they come, with the goal of building a delightful video experience for both Pinners and developers.

We’re building the world’s first visual discovery engine. More than 250 million people around the world use Pinterest to dream about, plan and prepare for things they want to do in life. Come join us!