New Narratives in Virtual Reality
Our studio downed tools for the day to look at new approaches and design considerations for virtual reality. This is the first of a series of articles documenting what we came up with.
Virtual Reality’s potential for immersive storytelling has been clear since its recent rebirth, with some of the first Oculus Rift development kits being used to create rich experiences for existing shows and brands.
The capabilities of these headsets and the devices that power them are progressing in leaps and bounds, and it’s fair to assume that projects such as Google Daydream, OSVR, Vive and Hololens will bring the experience to a much wider market in the coming year.
This wider market in turn means a larger demand for VR content. This content will have to be tailored to the virtual reality experience: In traditional video production, you don’t have to worry about the camera becoming sentient, turning its head and walking around on set.
Decide the role of the viewer
With a greater sense of immersion and interaction there is a much wider range of expectations your viewers will have coming in to your content. When sat in front of the TV there is a very clear line between video and video game — one is active, the other passive, one is video, the other a 3D environment, one is interacted with holding a TV remote, the other a game pad. In virtual reality the line blurs.
In light of this, it’s important to maintain a consistent level of interaction to the experience you build into your narrative. If a viewer has to interact after spending 20 minutes not doing anything, they may miss an important cue to action. If there’s a long segment of passive viewing during an otherwise much more interactive experience, the user may grow bored or frustrated trying to interact with static props and scenery, pulling them out of the narrative.
The framework we created defines three different categories or levels of interactivity for your experience:
If the viewer is an Observer, they are in a situation most similar to traditional movie goer or couch potato. The only difference is that rather than the fixed perspective of the movie screen, they become the camera, able to look around a scene as the linear narrative plays out in front of them.
The majority of 360 video being produced right now fits within this category, and it has the lowest barrier to entry in terms of production: The experience is simply that of a video player, with the only interaction necessary for the user being the ability to stop, start and skip forward or backwards through the video.
Beyond video a lot of 3D ‘narrative experiences’ being built in VR fit within this category. The player is simply a camera within a linear story, whether it was shot on 360 video or is real time CGI.
Given a little more control over the narrative and their place within it, the viewer is a Player. They can control the pace and flow of the narrative, perhaps to the point of being able to control the direction the story takes or the end result. This depends either on branching narratives or mixed media experiences that are much closer to being cinematic video games than traditional video.
Early examples of this category suit the medium of VR much better than traditional video players — it can allow for the user to experience a 360 world around them without necessarily having to know where to look to see where the action is or to advance the plot.
Finally, the next generation of virtual reality experiences place the viewer as a part of the narrative as a Character. Cinematic gaming and virtual experiences that could take advantage of open world experiences such as Grand Theft Auto or Second Life, or take on the mantle of Karaoke for your favourite films and actors.
This final category is purely speculative, leaning heavily on the ideas of modern Interactive Theatre and Immersive Cinema. Voice Recognition, Chatbots and Social VR could all be key technologies that drive these experiences.
Guide the user through the narrative
In traditional video production, narrative is guided by the composition of the frame: How characters, props and scenery relate to each other in terms of size, motion and position. Once again, in virtual reality the rule book is opened up. The viewer has control over what they look at whenever they like.
For a very open narrative or exploration of a space, this opens up a great amount of freedom to the viewer and the filmmaker to offer a range of different elements and motion for the viewer to focus on. However, it can also mean that it’s easy for the viewer to miss something important without adequate sign posts and hints. All of these tools already exist in the domain of traditional film production, but require new approaches in virtual reality.
The first set of tools the filmmaker has at their disposal is the same as traditional video production: Visual cues such as composition and motion can guide the eye. It’s difficult to miss a sign if it’s a foot away from your face and flashing on and off.
On a more subtle level, motion as simple as a dragonfly or tumbleweed could guide the user back towards a key visual moment in the narrative, or use of lighting to pull attention to a single spotlight’s focus. The use of visual cues has to be balanced with the style of the content itself — garishly glowing switches or arrows pointing to where to look are likely to pull the viewer out of the experience.
The next tool in the box is audio production. One of the biggest leaps made in virtual reality in this area is positional audio. By simulating the position of your head in 3D space, it’s very easy to ‘place’ the source of sounds within that space. This makes it straightforward to draw the attention of the viewer to a character that’s speaking, or a prop making a loud, attention grabbing noise.
This use of audio can be used to not only draw attention within the scene directly, but also with atmospheric audio or soundtracks to guide the users gaze. Looking away or towards the area you intend the viewer to look could fade in an ambient audio track or soundtrack that reinforces the viewer’s emotions or attention.
The final step to guiding the narrative is breaking through the fourth wall with Meta narrative. Like Ferris Bueller turning to face the audience or the ending of Blazing Saddles, characters or scenery can call out the virtual nature of your experience, guiding you towards taking action or the intensity of an experience that’s coming up.
If poorly judged, any of these interventions could very easily pull the viewer out of an immersive experience, highlighting its imperfections and the artificial guide rails. However, if handled well, the path laid out by an experience or narrative can be deliberate and invisible.
Build immersion beyond the visual
The hardware for virtual reality currently centers around the headset. Higher density screens often originally built for smart phones and tablets and a growing array of gyroscopes, accelerometers and photosensors attempt to work seamlessly to mimic vision and motion to pull the viewer in to the world they’ve come to experience.
These technical hurdles are the first steps to building an immersive experience, but they certainly aren’t the last. A lot of work can be done to create a believable world that pulls the viewer into the narrative regardless of what fidelity the experience is.
Perhaps the most important of these is creating a world that makes sense — that follows its own rules and internal logic. Chuck Jones’ characters on Looney Tunes often had rigid rules that defined they would act and react in any given situation, and this consistency led to a coherent world in which characters acted the way you expected them to and would be easily understood by children of all ages.
Within virtual reality, we have to consider the internal rules and logic of the experience we’re creating — what is the role of the viewer, how do we guide them through the narrative, how do the physics and interactions of the world work–before the viewer can disconnect from concerns or questions over how to go to the next scene or whether they should be waiting for something to happen. Once the viewer has stopped doubting the constraints of the world, they can fit their expectations and beliefs about what’s going to happen next or what to do.
Where to look next
The ideas that spawned this article came from a team of four designers brainstorming over the course of a single day. There is much more work to be done in testing and validating these techniques and approaches, and we will be sharing progress on these with the wider community as we go. If you would like to contribute to this process or want to build on our initial thoughts further, please get in touch or comment below!
Headquartered in London, we design experiences that integrate brand, product and services for the leading media companies around the globe. Our team of 50 designers, technologists and strategists translate complex business problems into simple design solutions from product idea to launch.
