Unframing — how mixed reality technologies are reshaping our perception
Key takeaways from Lighthouse’s Reframed 2017 conference on virtual and augmented reality.
As a branding agency, we talk a lot with our clients about not only finding the right messages for their audience, but the right context; answering the question, where will your audience be when they engage with your brand? The responses, often vast and nuanced, enable us to build a picture of the various assets and content channels for their business. But what if we could dismantle context? Teleport the viewer into a new environment, or even obscure their understanding of their own physicality? Through mixed reality, we add to where, and ask; who will your audience be when they engage with your brand?
Mixed reality technologies not only present the opportunities for businesses to explore new content channels, but create entirely new contexts for those engagements to exist. Perhaps the best way of explaining this is through the lens of the film, Notes on Blindness — a film and virtual reality experience depicting the narrative of a man losing his sight, introduced at Reframed by its producer, Landia Egal. Where the film itself augments reality by overlaying cinematic visuals onto the real-life audio diary of its central character, John Hull, the virtual reality experience enables viewers to then embody the character of Hull himself. In the cinematic film we remain ourselves, watching the story unfold through a 16x9 aperture; yet in VR we are asked to step into the body of Hull, his voice displacing our own. We’re in first-person shooter territory.
Listening to the line-up of speakers at Reframed 2017 it was clear that many were grappling with the implications of emergent VR/AR/MR technology on how we fundamentally engage with, and perceive, reality and our sense of self. It’s an interesting conundrum, as we witness the implications of social network technologies have had on reframing political discourse through the creation of polarised echo chambers whose algorithms perpetuate the homogenisation of content we’re exposed to. Where these social networks have enabled us to create evermore siloed tribes, mixed reality technologies enable us to feel and see those constructed realities. This new wave of technologies, from the Oculus Rift to Cardboard, to iBeacons and Magic Leap, are explicitly designed to create immersion; but immersion in what? By whose narrative, and fundamentally, for whose gain?
VR is the ultimate empathy machine
Undoubtedly, the Notes on Blindness VR experience enables a sighted audience to empathise with John Hull. We inhabit his body, his world, and through beautifully crafted visual interpretation of his words, begin to understand the small but significant differences in how we perceive reality. For Hull, rain lights up the world; the sound of water hitting the surfaces around him gives them presence in his — now our — world. The use of virtual reality as a machine to trigger empathy is hugely popular in our nascent content industry, with best-in-class examples including the UN’s Clouds over Sidra (a VR walk through a Syrian refugee camp). Though, as well articulated by Robert Wang in his poignant takedown, there is an often neglected responsibility we have as creators to question why we are using these technologies to engender empathy — and how authentic that empathy is by consequence.
VR challenges our perception of self
If we know that virtual reality can open our eyes to the experiences of others in a more embodied way than other communication mediums have been able to before, what can it achieve with our own perception of self? Multi-disciplinary artist and researcher, Kim-Leigh Pontin, discussed that the method for engendering empathy in VR is often through positioning the viewer within the narrative. No longer passive consumer, VR creates the opportunity to devise or expose a character for the viewer.
“The only reality we know is the stuff that we experience” Kim-Leigh Pontin
From the moment you invite someone to engage with a virtual reality experience, you have an opportunity to write them into the narrative of the content they are about to consume. At last year’s PLAY expo, the public were presented with the opportunity to ride a VR rollercoaster; but the experience itself did not begin in the headset, rather in the queue for the ‘ride’ itself. By implementing a ticketing and queuing system akin to riding a “real life” rollercoaster, participants began to mentally prepare themselves for participating in the experience. When your turn was called, you were sat in a traditional rollercoaster chair (though oddly displaced in the isolation of an exhibition gallery), the safety bar dropped on your lap, and you were instructed by the facilitator to keep your hands on the bar at all times. In virtual reality, whenever you looked down to where the real bar and your arms would be, you saw virtual arms. Your body, now in VR.
Flying through a snow scene, the creators of this experience had opted to portray the participant’s arms in a gender-neutral coat and gloves, no skin exposed: a smart move towards ensuring the sense of immersion was not broken for the majority of audience members.
VR replaces storyboards with non-linear experiences
Therein lies the rub. For virtual reality to work, for it to live up to the expectations set by science fiction, it needs to be immersive. This isn’t the garden variety of immersion; getting lost in a book or Netflix series, it’s not spending consecutive days on Second Life, it’s the utter suspension of disbelief. This kind of immersion is about transcending into the constructed reality presented to you.
So how do you create immersion? As Chris O’Reilly from Nexus suggested in the inaugural talk at Reframed, it is, in part, about giving the participant agency. The biggest challenge he found in creating the VR masterpiece Rain or Shine was ensuring the viewer could still follow the narrative, whilst still giving the VR medium purpose. In VR, rather than sitting back and watching a narrative unfold, you’re asked to explore an environment; but to tell a cohesive story you need to ensure that the audience are still exposed to certain pivotal plot points. In Rain or Shine, it’s imperative that you see the protagonist put on her sunglasses, and the VR experience is constructed so that the story will not progress until the viewers gaze has landed on the girl to watch this action happen. O’Reilly actually shared a fantastic matrix they used to plot out these key scenarios — mapping what a viewer might have seen, what they won’t have seen and the routes to ensure they are kept up to date with the narrative. For anyone who is familiar with edtech, this might sound very similar to branched learning.
It’s not only edtech we can adopt strategies from, but theatre and gaming. Throughout the day many of the speakers discussed how Punchdrunk theatre, a walkabout-style production house, guide their audience through the content whilst still conveying a cohesive story; using a mix of repetition, content placement and character crossovers. For storytellers to really grapple with the differences between cinematic storyboarding and VR’s exploratory narratives, Rob Morgan from PlaylinesAR discussed that you’d be better off looking towards the infamous tabletop RPG, Dungeons and Dragons. In creating VR narratives, you are placed in a similar position to an experienced Dungeon Master — tasked with designing a flexible, responsive script that enables the players to exact their own journey, whilst still striving towards a set destination.
We’re not there yet.
But it’s clear from the discussion amongst content creators and technologists alike that this is on the horizon. O’Reilly aptly said; “emerging technology is just the stuff that doesn’t really work yet” and he’s right. This industry is still so new that we’re having to work out the processes and intricacies of immersion from the ground up. Giving viewers a sense of perceived autonomy and agency over their own journey through your content is key to creating a compelling, immersive VR narrative, but this throws up huge ethical challenges in how we create that content; what worlds we decide to build and for whom they are created.
If we are to learn lessons from the questions of morality posed to us through decades of science-fiction depictions of VR, we need immersive content creators and technologists to engage openly in this dialogue and work towards creating a framework for mixed reality content to exist. As businesses and brands, we need to strive to see the opportunities in virtual reality beyond the gimmick, but do so with great conscience.
About Pixeldot
At Pixeldot, we’re working with our clients to create the brands and businesses of the future. If you want to talk to us about the speculative futures of your brand, or discovering your brand thread then come and book an office hours session with us for a chat.
