React native video plugin & thumbnail view

3 min readJul 18, 2024

Here is a small article to explain on how to integrate thumbnail view in a react native app using react native video and the new plugin system.

This has been developed together with TheWidlarzGroup. — React Native experts which maintain the library.

Plugin system

A new plugin system has been developed to facilitate custom integration of any feature.

Pull request introducing the feature is available here: https://github.com/TheWidlarzGroup/react-native-video/pull/3909

The idea of the plugin system is to be able to integrate third parties libraries using player handle (Exoplayer or AVPlayer). This plugin system is particularly useful to integrate analytics system. Most of analytics libraries use directly player handle to track player state, buffering statistics, …

So it is now possible to create a new package with native code which will be able to interact directly with the player handle without patching react native video.

Here is small architecture diagram explaining on how the handle is transfered to plugin and how plug can be registered:

Now you can get analytics plugin that were known from the commercial players like Theo (see Theo’s connectors hereunder). And if you need a plugin like, this reach out to TWG to get that done.

When developing an analytics plugin, user will also need to provide metadata. Custom communication channel shall be put in place to provide metadata to the 3rd party library you should develop your custom function like in the following sample.

Another possible use case we found is to integrate the thumbnail on the seek bar, this is the sample use case develop for the proof of concept.

Find a good sample

We had some difficulties to find a good sample of thumbnail view mpd file.

Some commercial player use a side file to get the thumbnails, some other use a custom vtt file.

I decided to use the reference video with thumbnail provided by dash forum: https://dash.akamaized.net/akamai/bbb_30fps/bbb_with_multiple_tiled_thumbnails.mpd

Exoplayer integration

Unfortunately this file doesn’t work work this exoplayer …

The issue is documented here: https://github.com/google/ExoPlayer/issues/9938

Dash forum provide a duration in flat format and in second instead of an integer value in millisecond

<AdaptationSet id="3" mimeType="image/jpeg" contentType="image">
   <SegmentTemplate media="$RepresentationID$/tile_$Number$.jpg" duration="634.566" startNumber="1"/>
   <Representation bandwidth="12000" id="thumbnails_102x58" width="1024" height="1152">
      <EssentialProperty schemeIdUri="http://dashif.org/thumbnail_tile" value="10x20"/>
   </Representation>
   <Representation bandwidth="24000" id="thumbnails_256x144" width="2048" height="1152">
      <EssentialProperty schemeIdUri="http://dashif.org/thumbnail_tile" value="8x8"/>
   </Representation>
</AdaptationSet>

The best solution has to be able to build exoplayer from source and then to patch it.

Following PR includes the build from source feature: https://github.com/TheWidlarzGroup/react-native-video/pull/3932

New Plugin development

The sample plugin is available here: https://github.com/TheWidlarzGroup/react-native-video/pull/3943

The plugin is simple:

UI update

You can check the video in the ticket to see sample UI display in the sample.

I just add a thumbnail view over the seek position.

The reported image is the following, It includes multiple sub images, so I had to calculate correctly how to display it.

A sample video can be found in related pull request:

https://github.com/TheWidlarzGroup/react-native-video/pull/3943

Conclusion

This proof of concept allows to get more flexibility regarding to react native video customization. Plugin system allows to integrate analytics or to develop custom feature dedicated to niche use case.

Building from source allows exoplayer debugging inside the fully integrated react native app.

Now react native video users shall have all keys to extend and develop their own features linked to the player ! :)