Interactive Video - Example Projects
Here are the example projects mentioned in my article about Interactive Video. The selected projects are used to exemplify the different ways to use interactivity togheter with video:
- add functionality: Sons Of Gallipoli
- make it richer: Canada Goose — Out There
- make it respond: Outcast — The Possession Begins
- make it your story: Nissan Infiniti — Deja View
You don’t have to read the previous article for this to make sense but it does provide some context.
Sons Of Gallipoli
Sons Of Gallipoli is an interactive documentary were the digital production was done by Jam3, we helped out with the interactive soundtrack. This is a perfect example how adding functionality can make the experience much more rewarding for the user.
The video content is a one hour long linear film living on YouTube. The interactivity is added as a layer on top of the video. It adds functionality to the video that simply is not possible on the YouTube platform:
- chapters, table of content and navigation
- extra material about the current topic in the film
- a map which gives you the location of the events
- sharing of specific parts that interests the user
The result is a much more rich and rewarding experience.
The original video is not split into chapters. The chapter cards are added on top while pausing the video. Since the audio of the video stops when paused we added a sound that covered this pause and made the transition seamless.
By adding chapters and navigation the content is easier to grasp, like looking at the table of content in a book. The investment in time for the user feels smaller, you don’t have to commit to the full hour, you can just watch a chapter, take a puase and continue at a later point.
At certain points in the film the user can access additional material, images and text, about the current subject in the documentary. Integrating it with the video puts the material into context. The user can learn more about what interest them which make the experience much rewarding.
You can try it yourself here.
Canada Goose — Out There
Out There is a great example of how interactivity can be used to make the experience much richer. It is the first ever global campaign released by luxury outdoor brand Canada Goose. It features the real, incredible stories of the brand’s most inspirational brand ambassadors. The interactive short film was shot in New Zealand by an impressive veteran film crew. Directed by Academy Award-winning filmmaker of “Crash” and “Million Dollar Baby” Paul Haggis, cinematography by “12 Years a Slave” Sean Bobbitt, music by Oscar-nominated composer Josh Ralph and sound engineering by Oscar-nominated Tom Myers/Skywalker Sound. CAA was our client that created the concept, we did the digital production (no Oscars so far).
The backstory
It is not every day you get the chance to destroy the cinematic art of a two time Academy Award winner with some interactive disruption but that was clearly a possible outcome here.
The film was surely going to be spectacular, no doubt. But it was the idea behind the script that made it truly special. Each of the scenes in the film is based on real events in the life of the brand ambassadors.
They were interviewed about their memories of the specific event, and they told this story and several more in a very personal way. They also allowed access to their private photo albums with previously unpublished images. All ingredients were there to add emotional depth to the scenes in the film.
Normally this kind of material is tagged on in the end but then we would miss the opportunity to give the scenes that extra depth. Instead we wanted to integrate it and connect it to the actual scenes. When the user reaches a scene they should get the option to go down the rabbit hole to hear the real backstory and see the images.
It was essential to not break the movie magic so the integration had to feel like it was a part of the film. The transition must be totally seamless, the music score should continue uninterrupted, the visual transition and the presentation of the personal photos must feel cinematic and in the style of the film.
Here are some early drafts for the photo albums
How it turned out
For the user it starts like any normal video. When moving the mouse/touching the screen a timeline appears at the top which allows the user to navigate to any position in the film. The rabbit holes are marked on the timeline so that the user understands there is something special happening there.
When the user reaches a scene a subtle indicator shows that they can experience the true story behind the scene. By interacting the user is seamlessly taken into a photo contact sheet filled with personal photos describing the scene. The person starts telling their remarkable memories with their own voice, while photos from their personal collections visualises the story. In the site capture below you can hear Marilyn Hofman-Smith tell the story of her close encounter with death.
There are two additional memories that the person shares. The contact sheet is interactive so the user can explore it and find additional photos by moving it around. When the user is done they are transported back to the film at the same position where they left off.
It doesn’t feel like the main film is interrupted, it feels like the story is enhanced. From this point the user looks at all the fictional scenes with totally new eyes, knowing that there is a real story behind the scene.
You can try it yourself here.
Outcast — The Possession Begins
This is an interactive trailer for the TV horror show Outcast that we produced for our client Campfire. The series is created by Walking Dead Robert Kirkman and is all about demons and possession, kids chewing off their own fingers and worse, great fun!
The experience allows you how it feels to get possesed. When you close your eyes you allow the demon to possess you, when you open your eyes you realize that the possession is all around you, there is no escape. It is all about immersion, to bring the user into a narrative and lose themselves. It is hard to find an example more suitable of an experience that responds to you!
Some backstory
Budget and time did not allow to even think about developing a custom solution for detecting opened and closed eyes, so we looked at which existing solution would produce the most reliable result. Visage was by far the best at the time. However, the library was huuuuge (13MB) so using it on mobile devices felt like a bad idea. This was not a showstopper since mobile is used in situations where closing your eyes might seem suspicious to people around you.
While keeping your eyes closed you could hear a demon walking around you (3D binaural sound, headphones highly recommended). The visuals weren’t that important (eyes closed, duh!), we made a simple video loop with the black goo that is so persistent in the series.
The demon script was a linear story which you followed to the end, even if did not want to. You could try to your eyes to stop it but the moment you close them you were back exactly were you tried to escape.
For the open eyes world we needed to create visual material which required access to original material from the series. This turned out to be a challenge. The fear for leaks before launch of the series was palpable (the general paranoia for leaks has gotten far worse since then 😓). We were required to work on a computer not connected to the Internet, not a simple thing interactive production.
It was solved through a combination of physical security and that we only got access to one second long clips that did not reveal ANYTHING about the plot. In Swedish we have an expression which translates to “cook a soup on a nail” which describes the situation perfectly. The soup actually ended up quite tasty in the end.
Depending on where you opened your eyes in the demon storyline you were served clips that were connected to what the demon was talking about. The clip ended in a loop (seamless of course) until you closed your eyes again.
The interactive soundtrack was a very important part in making the experience immersive and as intense as it was. I will cover the importance of sound in interactive applications in a future article.
From a technical POV
The experience was made to have the widest technical support possible. It supports desktop, tablet, mobile, and can be controlled by eye tracking, keyboard control, touch. The full experience is supported on all major browsers including Internet Explorer. iPhone before iOS10 doesn’t support interactive film since it opens video in its own player, but thanks to a hack found by our friends at Earth People we were able to support that too.
And how did it turn out?
Pretty good I think. It won three Clios, a gold for Technique, a silver for Innovation and a GRAND for Digital, and a Bronze Lion in Cannes for User Experience Design. It is not an Oscar but close enough (genetically Lions and Oscars are 99% the same, I don’t know about Clios though).
You can try it yourself here.
Nissan Infiniti — Deja View
Deja View is a great example of choose-your-own-adventure. Itis an interactive film where you interact with the characters in the film by talking to them. During the experience the characters will pick up their phones and call your real world phone. What you say to them determines what will happen next and how the story evolves.
The backstory
When I read the brief from our client Campfire for Deja View it was immediately clear that this was something out of the ordinary. It described an interactive film experience where the characters in the film calls you on your mobile phone and you have a conversation with them. It had to respond immediately to user interaction. The goal was clearly to create a motion pticture quality experience. At the time I did not know the scope of the script but in every aspect it felt huge, lots of scenes, alternative takes, clips.
Super cool but could it be done? Or rather, could it be done with the quality and user experience that the idea deserved? I jumped into the deep end to define the project components. A custom phone system, speech recognition, natural language processing, dialogue flows, the list goes on. And the icing on the cake: realtime edited interactive video with a level of control never doen before. It was a fantastic challenge!
The producer expected a Flash solution but I did not want to do that. For me it was already dead as a technology (this was 2013, ancient times). I also wanted the experience to run on iOS and Android. This was not a requirement from the client, but the mobile trend was clear to me and it felt like a good investment for the future.
The traditional method of preloading clips would not be feasible. At all decision points there would be several clips to preload since all had to be ready for playback. The logic for this would become very complex and brittle. Adding tablet support and it was clear that this was not the way to go.
Without gettting too technical we developed a method tthat allowed us to use a single video element and that supported the unique complexity of the project.
But that was only one of many challenges. The one that took the longest to solve was the phone system, the speech recognition and natural language processing. It was not until a couple of days before the deadline for the proposal that we knew we had a solution that could deliver with quality. One day that story might get its own article.
How does it work?
The user calls a number and gets a code, enters this in the browser and the experience starts. The first scene show the two main characters sitting in a car, waking up with no memory about who they are or where they come from. Looking at their mobile phones they discover that someone has called them several times, the number on the screen is YOUR phone number. The man decides to call you and your mobile rings in real life. Here is a screen capture:
The characters calls you at three different times to ask you for help to solve the mystery. During the calls you don’t see the actors (alhtough that would have been possible too), what you say during the calls affects how the story evolves.
In the screen capture above the user mentioned “dry cleaner” (makes sense if you have done the experience 🙂). After the call we weave what the user said into the conversation by playing a corresponding clip.
The video is very responsive allowing us to change a single sentence as above whithout any latency, buffering and loading. In several other decision points there are up to four alternative clips, sometimes within a second from the user interaction, which are played back as if it was edited that way.
There was over 35 minutes of video and a total of over 300 clips that were controlled interactively. Below is a simplified flowchart for the video part.
The flowchart does not include flows for the interactive audio track. To cover all these edit points the audio could not be on the video itself since that would cause ugly cuts in the audio. Creating an interactive sound track is an essential part of making the experience feel like motion picture quality. I will cover this part in a separate article.
The flow charts for the interactive phone calls are also not there, each of them similar to the one for video, a potential topic for another article (so much to write!).
You can try it yourself here.
This was part two (three?) of the video series. Here are all articles currently published:
Forget about VR, AR, AI and ML: Video is the Next Big Thing!
Interactive Video: Why and How
Interactive Video — Example Projects
Personalized Video, dynamics and a fancy word for “targeted”
I will add new video related articles now and then.
Before you go
Clap 👏 👏 👏 5, 15, 50 times if you enjoyed what you read!
Comment 💬 I’d love to hear what you think!
Follow me Johan Belin here on Medium, or
Subscribe to our newsletter by clicking here
