How to display animated GIFs on Roku using SceneGraph
For a while now, YouTube has had GIF previews for video thumbnails with the intent to improve the user experience. Users can easily see at a glance a short clip of the video without having to play the actual video, saving the user a few clicks.
Netflix also makes use of animated GIFs, albeit not as widespread as YouTube. They seem to use GIFs solely as a promotional tool, just to capture the user’s attention for specific content.
GIFs have been around forever, but lately they have been taking over OTT apps as a great way to engage with users.
Big players like Netflix and Youtube have access to Roku’s NDK, which gives them way more freedom and control over the hardware. Most Roku developers have to work with BrightScript and SceneGraph, which are very underpowered tools, specially when it comes to tackling big UI challenges.
Helpful tip:
If you’re curious to know which apps use Roku’s NDK, you can use the External Control Protocol and fetch all the installed apps on your device. It will return a xml with this structure:
subtype="ndka”.The SceneGraph solution
If you look at the Poster node documentation, it states that “The Poster node class supports JPEG and PNG image files.“. That’s not a 100% accurate, because the Poster node also supports GIF images, but, it cannot animate those GIF images. So for the case of animated GIFs the Poster node will only render the first frame of that GIF.
The fact that SceneGraph does support the GIF image file format is a big step towards our goal and makes evident the solution to displaying animated GIFs.
- Extract the frames of the animated GIF and store them as individual GIF files.
- Create a frame-by-frame animator using a Timer node.
- Display each extracted frame on a Poster node using a frame-by-frame animator.
Extracting the frames of a GIF image using plain BrightScript + SceneGraph is a bit of a challenge since Roku does not provide any image manipulation APIs with their current SDK, so the only real way to approach this problem is by going straight to the file and process its bytes and bits following the GIF file spec. The main component for this job is roByteArray.
Let’s get to the good part — the code 😎!
Prerequisites
Before we continue, there are a few things that we need to cover to be able to run and test the demo app.
- Roku device (a Roku 4 or above is recommended).
- Developer mode enabled on the Roku device.
- Knowledge on how to side-load a Roku application. There are many ways of doing it, you could use the browser or an Atom plugin or a Sublime plugin or your own solution, it doesn’t matter.
- Knowledge on how to access the BrightScript console.
Starter project
Download the starter project and run it. You should see a simple interactive grid with Poster nodes in it:
The Poster nodes are displaying only the first frame of all the GIFs available in the bundle, we are going to fix that and make sure those GIFs animate.
The GIFs are Frozen-themed so I’m sorry if you let it go already 😉.
Decoding the GIF files
As mentioned earlier, we have to extract the frames of the GIF file so that we can display each frame sequentially using our frame-by-frame animator and give the sense of motion.
GIF files are composed of 11 different types of blocks of data:
There is a lot of information out there that covers the specifications of each block type in great detail (references can be found below), so I’ll just do a quick run through of each block type, explaining how we’ll use them, covering only the essentials.
- Header: Marks the beginning of the file, is always 6 bytes in length and is composed by the Signature (always “GIF”) and Version (“89a” or “87a”). Our implementation will always ensure that the header is “GIF89a”.
- Logical Screen Descriptor: Defines the whole GIF displaying boundaries. Always 7 bytes in length, we only care about the 5th byte which is a packed field that helps us determine if the GIF has a global color table and the size of such table.
- Global Color Table: Sequence of bytes representing red-green-blue color triplets. If present, we only need it to prepend it to the extracted frames that do not have a local color table.
- Graphic Control Extension: Contains parameters used when processing the image data that follows, particularly the delay time, which we will read to calculate the average delay time between all frames. Has a fixed length of 8 bytes.
- Image Descriptor: Defines the image boundaries and helps us determine if the image has a local color table and the size of such table. Is always 10 bytes in length and the table availability info is stored in 10th byte.
- Local Color Table: Same structure as the global color table. If present, we just need to keep it attached to the image data.
- Image Data: LZW encoded sequence of sub-blocks, of size at most 255 bytes each, containing an index into the local or global color table, for each pixel in the image.
- Application Control Extension: Always 19 bytes in length. We don’t need this block in our extracted frames, we just need to skip through it.
- Comment Extension: We don’t need this block in our extracted frames, we just need to determine where it starts and ends and skip through it.
- Plain Text Extension: We don’t need this block in our extracted frames, we just need to determine where it starts and ends and skip through it.
- Trailer: Marks the ending of the file. Is always 1 byte with the hex value:
3B.
Based on those block type descriptions here is the complete implementation of the GIF decoder component that we’ll use.
Please add those files to your components folder and go over all the comments in the BrightScript file for better understanding of the implementation.
Our new GIFDecoder follows a delegate pattern that we’ll use to communicate with the scene, so let’s set it up. Go to the AppScene’s init() method and create a GIFDecoder instance at the beginning of the function.
Now let’s declare and implement the GIFDecoder callback function which will be executed after the decoder finishes. To declare it, we just need to add it to the XML interface.
Moving on to the callback implementation, we’re just going to print out the new frame urls for now.
And finally, we have to actually start the decoder for the focused poster.
If you run the app, this time you should see all the urls for the extracted frames appear on the BrightScript console as you move through the posters.
Animation time! 🎸
We’ve reached the final and easier step of our solution, the animation 🙂. For that, we’ll simply use a Timer node to display each frame sequentially but we’ll wrap it within our custom frame-by-frame animator component to keep track of each frame more easily:
Please add FrameAnimator.xml and FrameAnimator.brs to your components folder.
Lastly, we just need to plug in the new FrameAnimator component on our scene.
- Instantiate the
FrameAnimatornode.
2. Stop the animator when navigating to a new poster.
3. Start the animation after the decoder finishes.
Side-load the app one more time and navigate through the posters, the focused poster should display the animated GIF 🤞.
If you are seeing the focused poster animate, CONGRATS!, now you can have animated GIFs on your SceneGraph apps 🍾.
The complete project can be found here.
Missing features
- Inter-frame coalescing: Some GIFs may have frames with empty or transparent areas that are meant to be rendered on top of the previous frame(s). Our current implementation displays each frame individually for performance reasons, GIFs that require inter-frame coalescing will not animate correctly.
- Variable frame rate: Some GIFs may have different delay times between frames, our implementation ignores this and uses the average delay time as the frame rate for all frames.
Conclusion
GIF thumbnails can be a great tool for improving the user experience in your app. They can be used to promote content or to offer quick content previews. But, as we all know, Roku devices are not very powerful (and they are not trying to be), specially with the tools available to most developers. A feature like this is obviously very expensive, so please use it responsibly 🙂.
