Setting Expectations For VR
Did you know that virtual reality is not a new technology?
The year of 2016 saw a huge increase of interest in VR. First of their kind VR systems have been released for consumers by Oculus, HTC, and Sony. Google followed up with the Daydream headset in addition to their already low-cost and popular Cardboard viewer. And leaders within the entertainment industry of the likes of Pixar, ILM, and Disney are experiencing an exodus of talent — whom are all leaving in pursuit of VR.
What is all this about?
A Brief History
The following is a summary of the first part of Evan Suma Rosenberg’s lecture — “Making Small Spaces Feel Large: Practical Illusions in Virtual Reality”
VR is not a new technology.
In 1838, the first “apparatus” to create a stereoscopic 3D phenomena was published in scientific research. The first VR “simulator” with immersive multi-sensory experience was created by Morton Heilig in 1956, known as the Sensorama. And the first VR “head-mounted display” was created by Ivan Sutherland and Bob Sprout in 1968, named the Sword of Damocles (it was tethered to the ceiling). Then NASA created a wearable head-mounted display in 1985, while in the same year Jaron Lanier founded the first VR company (VPL).
Yet VR didn’t first enter the public’s attention until Scientific American published an article on VR in 1987 and a sci-fi horror film was made in 1992.
There were valiant commercial failures such as the Sega VR in 1993 and Nintendo’s Virtual Boy in 1995. But the big shift happened in 2010, when head-mounted displays were capable of bringing forth a 140 degree of field of view to create the first magical user experience — it was night and day in terms of experience differences compared to previous prototypes.
What further popularized the technology though, are smart phones like the iPhone. Their small and high resolution screens had enough computing power to enable mobile VR systems. And that helped Palmer Luckey to found Oculus VR and later be acquired by Facebook for $2B in 2014 — the same year Google released Cardboard. Fast forward to today, we have OSVR, FOVE, HTC Vive, Oculus Rift, the Sony PSVR, etc, and indie and major developers making games/film in VR.
But we still don’t know what VR is for.
Is VR worthwhile as a new digital medium according to the current commercial hype?
Will this new paradigm disrupt existing digital paradigms and affect the way we play, relax, socialize, learn, and work in the near future? (Douglas Bowman, User Experience in the (New) VR and AR Revolution)
To answer those questions, I had to acquaint myself with the existing body of knowledge on VR. On top of the research papers I read, here are some noteworthy resources that I have gone through since August:
- Vincent McCurley’s “Storyboarding in Virtual Reality”
- Matt Sundstrom’s “How to Design for Virtual Reality”
- LeapMotion’s “VR Design Best Practices”
- Jean-Marc Denis’s “From product design to virtual reality”
- Jessica Brillhart’s “In the Blink of a Mind-Engagement, Part 1”, “In the Blink of a Mind-Prologue”, “In the Blink of a Mind-Attention”, “Virtual Reality’s Fundamental Question”, “How to Greet a Rebel: Unlocking the Storyteller in VR”
- Katy Newton and Karin Soukup’s “The Storyteller’s Guide to the Virtual Reality Audience”
- Dr. Janet Murray’s “Not a Film and Not an Empathy Machine”
- Dr. Brenda Laurel’s “What is Virtual Reality?”
- Oculus Story Studio’s blog
- Mike Alger’s “VR Interface Design Manifesto”
Additionally, by privilege of Georgia Tech’s industry connections, I was able to participate in thoughtful discussions with VR designers and developers from leading technology companies and VR studios:
- Mike Geig, Technical Evangelist at Unity Technologies
- Kelly Braun, VP Research & Operations at Baobab Studios
- Yelena Rachitsky, Creative Producer, Experiences at Oculus
- Anthony Batt, CoFounder of Wevr
- John Bernhelm, Senior Experience Designer at Oculus
Thanks a lot for everyone’s time!
“Here is what I learned: VR must explore and establish its digital conventions as an emerging medium.”
The more information I examined, the more I realized how little I knew how VR should work. There are studies that have been done on what levels of realism is needed for a good user experience, how to create spatial illusions to make small spaces feel large virtually, and tested and tried methods to deal with locomotion for a room-scale virtual playscape. Yet despite how much research has been conducted or how many games are currently being produced, much is still uncovered. This was especially true when you want to do things that has never been done before, such as guiding the interactor’s attention in virtual reality.
It was very overwhelming.
But I didn’t want to admit defeat. The only feasible solution forward was to actually build a prototype to give shape to my conceptions of VR, instead of continuing to conjecture in the abstract. My thoughts were that a real application will put most, if not all assumptions that were slowly forming in my head to test.
Therefore, under the guidance of Dr. Janet Murray (who heads the eTV lab at tech and is the author of “Hamlet on the Holodeck”), I joined a team to create Lycan: a virtual reality project that transforms your identity via an interactive werewolf narrative. I will first explore how VR can be used to tell stories before doing anything else too ambitious.
And so for the last 3 months, my team brainstormed dramatic scenarios, critiqued existing VR games, films, and applications, mocked up immersive prototypes, and scripted our own storylines to explore intriguing interactive narratives. Most importantly we had to unlearn our crafts for the new medium.
It. was. a. journey.
First and foremost, it was clear that legacy media techniques do not apply. For example, voice-over, cinematic titles, editing cuts only serve to diminish a VR experience — because they magically appear for non-VR reasons and are thus distracting.
Secondly, we encountered numerous hardware constraints. In the past, VR systems have only succeeded in niche applications such as training simulators, phobia therapy, and basic 3D modeling, etc. Now the latency, rendering, and modeling resolution have all improved significantly, but haptic feedback and methods of interaction that are key to an interactor’s presence are still limited. Moreover, motion sickness was an issue we could not ignore. A survey we conducted during one of our prototype’s demo had 36.8% of participants report eye strain, 26.3% report difficulty focusing and having a blurred vision. Not to mention a few cases of mild difficulty concentrating or general discomfort.
Third, it was our narrative that brought the people and the technology together.
The emotional impact we were able to produce with our interactive narrative was profound. Most of our demo participants reported to be involved by our albeit crudely crafted virtual environment, and many more responded to the audio and visual cues we intentionally put in. And thanks to our use of natural metaphors from the real world, some participants instinctively started to move themselves around by walking, unprompted, once immersed in our virtual environment. This meant that interacting with our virtual environment was not an ordeal at all for our participants — we were careful to not introduce overly engineered digital conventions for tasks such as selection and manipulation.
Last but not least, we exploited dramatic agencies for our interactive narrative and utilized threshold objects to yield satisfactory results in terms of immersion and presence. Our participants reported being moved, surprised, and intrigued by feelings of suspense. It was elating!
The following video describes in full our design work.
Current technology is limited, and will continue to be for a while before VR can hit an amount of sensory fidelity that is close to the real-world. Yet we found that VR narratives may move forward their story beats with dramatic interactions regardless — giving us hope that within virtual reality stories can indeed exist. And Lycan serves as an experimental proof of concept.
Unfortunately my team and I ran out of time to prototype all of the narrative scenes we had conceived for our short 3 month schedule. If more time was given, we would have liked to also explore how exactly emotions can move forward interactive narratives; or how interactors can connect to characters and objects with subtle interactivity that enhances the narrative experience; or how much we could break away from reality and not be punished experience-wise; or how to best allow interactors to learn the rules of your virtual environment without instructions; and most importantly, what story genres can be best told within VR?
And that is not an exhaustive wish list.
My advisor Dr. Janet Murray states, “to invent a new medium you have to find the fit between the affordances of the co-evolving platform and specific expressive content — the beauty and truth — you want to share that could not be as well expressed in other forms.”
For sure, interactivity appears to be an essential element of VR, as evident by the burgeoning of VR games on current VR platforms. But in terms of content, legacy media techniques do not apply (from our project experience). My question now is: how may we leverage VR to produce creative content that is interactive in real-time, without sacrificing immersion and presence? That will be what I would like to explore next moving forward.