What is Reality?
Transcript of my keynote speech at the fourth annual Silicon Valley VR Conference.
Good morning. It’s a delight to see friends old and new at the fourth annual SVVR. For many of us, this is where the journey in virtual reality began. A small group of dreamers, holding fast to a future vision based on half-working technology whose success was by no means assured. From the first meetups and the Oculus Kickstarter, to peak hype, to where we are today, it’s been a ride.
Back in 2012, I tried on a DK1 for the first time. I gutted it out for 10 minutes, then took it off, woozily declaring: “not ready.” Too big, too bulky, and decidedly not cool. A veteran of the field, I stayed close to it, cautiously optimistic — but battle-scarred enough to not go all in. Like many of us, the Facebook Oculus acquisition was the watershed event that got even the most skeptical and wizened of us to pay attention. Maybe this time, I began to think, consumer VR could actually work — backed by a communications giant with ample resources and ambitious growth plans.
Just a few months later, I was invited by Karl Krantz to speak at the first-ever SVVR conference. By this point, I had decided, I was all-in: VR, make or break; this time, it’s going to work. I began to cook up the first of several startup ideas, and started advising and investing in companies. And stayed on the front lines as our young industry began to take shape.
And now here we are, three years later. The crazy bet seems to be paying off. VR is going strong, showing signs of continued growth, a diversity of uses and even the glimmer of the first killer apps. We have a thriving ecosystem of hardware, software, tools, big companies and startups, studios, independent creators, and enterprises continuing to push the bounds of what is possible with this evolving technology. We even have career specialists in something called “VR/AR Strategy…” whatever that is.
With this background in mind, and with another exciting SVVR conference about to kick into gear, I thought it would be good to step back and take a look at not only where we are, but what we are doing. And where we can go from here.
By the numbers: where we are is 5 million Gear VRs. 1 million PS VRs. Healthy sales of the desktop systems in the hundreds of thousands. Job Simulator on Steam grossing $3M. VR hitting the mainstream consciousness appearing in TV shows, commercials during sports playoffs, and my favorite VR image of all time:
Here’s 44…. deep in the Metaverse.
And so, it would appear, we have arrived. But have we actually crossed the chasm, from enthusiasm and early adoption on the way to mainstream use? Only time will tell. I don’t have a crystal ball and god knows I’ve been wrong before.
At my company, Unity, we’re bullish on the long term but quite conservative on the short term. We believe that investments in developing immersive content will pay off eventually. But we try not to let our enthusiasm for the potential get ahead of the reality. We are working in lockstep with VR creators in many industries beyond gaming and entertainment. Our struggle are your struggles. We’ll learn together and with luck, prosper together. In a moment, I’ll share some great stories, and insights we’ve gathered thus far.
But first, I would like us all to ponder a question. In the wake of recent geopolitical developments, and the pivotal role technology played in their unfolding, I have found myself asking on more than one occasion:
What is reality?
What is reality, when burst fire laid down in rounds of 140 characters at a time can challenge our most trusted, time-tested institutions, and lies, no matter how brazen, if told repeatedly, can get a major segment of the population questioning the very nature of what is real?
What I tell you three times is true. The neurolinguistic programmers architecting today’s dark revolution know this all too well, and are using it to their advantage.
If something like that can be unleashed using crude tools like social networks and cable TV, what can be done with a medium as powerful as virtual reality? When we can not just bend words and pictures, but all of our senses?
What will happen when we can use VR and haptics to deliver messages straight to the hypothalamus — even implanting memories? Obviously we don’t need to worry about this at scale yet… there aren’t that many Vives in the wild, and the experiences are opt-in and high touch. Nobody is going to voluntarily download a Steam app that turns their world upside down.
But imagine a near future — and I think most of us do — where the VR stack is ubiquitous, deep experiences are commonplace, and we’re plugged in hundreds of times a day the way we are with our phones. We’ll not only work and play in VR — or at that point more likely XR — but we’ll be communicating, socializing and getting our information this way. In this future, VR is the Web, the Web is VR, and the Metaverse is a machine not just for transmitting memes, but entire experiences. Straight to the brain. With much lower friction — see it on your feed and jump right in. Join our clan, hunt and kill those people… because they’re not real… right? Or even if they are real, it’s OK because they’re not us; they’re the *other*.
We’re all familiar with the concept that technology is value neutral: it can be used for good or ill. This holds true for virtual reality of course. But what is happening now in the world is a stark admonition. What we do next matters. A lot. In the months and years ahead, we have a dual responsibility: to grow a young industry, and to use every tool at our disposal to make the world a better place.
And now that I’ve gotten *that* off my chest…
Virtual reality — as opposed to the other kind — gives us new tools for improving our lot. I’m not talking about head-mounted displays, motion tracking hardware, stereoscopic rendering, haptic inputs, machine learning and all the other technological trappings. Those are the enablers, the platform tech underlying this new medium.
No, the tools I’m talking about are communication tools: empathy, escape, embodiment, and engagement. We can use these tools to design immersive experiences that have lasting impact… maybe not solving the world’s problems but at least making positive change.
Let’s explore these now.
Empathy
Much has been made about the power of VR to create empathy.
In a TED talk that launched a thousand startups, Chris Milk famously called VR “the Ultimate Empathy Machine,” citing the medium’s ability to change our perceptions of each other, and to close the gap between the story and the viewer. With television, computers, and phones, there is a space between you and the action. VR takes the story off the screen and into your own eyes: you’re not just a bystander to the ongoing conflict in Syria; you’re experiencing it firsthand. With VR, there is no gap: you’re in the story. You’re no longer a viewer, you’re a witness. You’re Within.
Emblematic Group, Empathetic Media, and other dedicated new media journalists use real-time CG to tell documentary stories in VR. These stories of war, family conflict and abuse of authority make us uncomfortable in a way that no lean-back 2D experience could, by placing us at the center of the action.
Construct Studio takes empathy a step further — completely obliterating the division between the participant and the story. In The Price of Freedom, you are the protagonist in a spy thriller based on the real world drama of cold-war espionage and mind control experiments. Spoiler Alert: it does not end well, and you may never think about our world the same way again.
Empathy helps us understand each other, and our world, in a deeper way.
Escape
If you’re like me, your capacity for self-enlightenment only takes you so far. It’s important to understand the world around us, but sometimes you just want to escape. Books, movies, TV, board games, puzzles video games… we use our communication media to thrill and delight, to tell stories of a brighter future, or sometimes, just to allow us to get off this rock for an hour or two.
Baobab Studios’ ASTEROIDS continues the space-bound adventures of Mac and Cheese. In this version, you play the part of a robot servant, helping the heroes through various tasks and, ultimately, saving the day. ASTEROIDS is feature-film quality computer graphics, but in real time running on a PC with an Oculus Rift, and YOU are at the center of the action. ASTEROIDS just might be the start of the interactive VR story for real. And it’s a delight to behold.
Felix and Paul’s DREAMS OF “O” is without a doubt the best 360 piece I have seen to date, based on Cirque du Soleil’s breathtaking aquatic show, “O.” For more than ten minutes, I was taken through a haunting world of water, fire and acrobatics. The tour guide: an white-faced bishop waving a brazier way too close to my face. I was terrified and mesmerized. I lost myself. No distance. No time. Just pure wonder.
The best VR entertainment creates a new world for us to inhabit and takes us on a journey through it… taking to worlds we didn’t even know existed. VR can bring us out of the mundane, even for just a short time. As much as VR is an empathy machine, I think it could also be the ultimate escape pod. And we could all use some of that… maybe now more than ever.
Embodiment
When we talk about VR, we talk a lot about presence. To date, the practice of creating presence in VR has been about establishing a sense of place that you feel is real… but the sense of self within that place hasn’t been there. So while you’re a witness, embedded in the action, it doesn’t really feel there’s a you there. Most VR feels like a third person experience, and creators who have tried to create a sense of self have often failed — mostly because we still don’t have good ways to represent our own bodies.
Life of Us, from Within, takes a novel approach this problem. It doesn’t give you a body; it gives you several bodies.
In this story of all life on earth, you evolve from protozoa, to a fish, to a primate, to human and even post-human. Life of Us uses positional tracking to great effect — you are essentially on rails so you don’t need to figure out how to move — and gives you enough agency with touch controllers
so that you could flap your arms to fly, wave to your companion, and draw streams of light in the sky.
It’s a totally great and fun ride, but where Life of Us really blows the lid off is in taking presence to a whole new level. It provides a live companion on the journey, so that we can feel ourselves through another. This goes beyond presence to embodiment. By taking the guesswork out of operating your body, and giving you someone else to see yourself through, you feel like you’re not only there, but inhabiting a true self. It’s a breakthrough in interaction design, and may just point the way for future applications in storytelling, education, training and social interaction.
Engagement
VR is already being used to market and sell products. And it’s going to be used to do that in the future. A *lot*.
As will AR and every other R. Every inch of available virtual real estate will be up for sale for manufacturers and marketers to hawk their wares.
The history of advertising is, after all, the history of a rise in production value. And with VR we so much more production value at our disposal: 3D graphics, animation, 3D sound, and haptic inputs, to name a few.
Immersive technology is going to be the ultimate marketing tool, because all of these goodies enable us to fully engage the user in an experience with all the senses, and allow them to actively participate. UK-based REWIND Studios recently created a multi-sensory, multi-user experience that puts you in the driving seat of a Formula E racing car, taking you on a thrill ride and highlighting the vehicle’s new features. Had fun? Good… maybe you’ll buy the car. But even if you don’t, your opinion of that manufacturer most likely went way up, and so did their brand.
All these deep interactions also result in way more data. Beyond the conscious interactive choices that the user makes, we can monitor direct manipulation input, track the head and eyes, and analyze sentiment based on subtle movements. With immersive technology, we’ll glean insights into individual and collective behavior even more than before. We can design better products, sell more of them, and better satisfy customers.
Now, these same techniques can be used for a lot more than sales and marketing. Engagement is key to the future of learning. The more actively involved we are, and the more physically engaged, the more we retain.
Everything from education, to situational awareness training, to operating machinery is going to get a huge bump from immersive technology. And the data that can be collected will help the educators and trainers develop better materials, in a virtuous circle.
The Future
These four tools — empathy, escape, embodiment, and engagement — give us a broad base to work from, to design immersive experiences that have lasting impact. They can be used to tell stories, about the real world or made up worlds; to teach; to ease pain and suffering; to sell products and build a deeper relationship with customers; to better relate to each other and the world around us.
VR is the most powerful medium we have devised to date. It can be used for good or ill, but in the balance I’m hoping it’s the former, and that collectively we can help safeguard our future. As technologists, scientists, educators, designers, storytellers, and entertainers, we do what we do to have fun, and to make money, but ultimately we do it to change the world… for the better.
Thank you and enjoy the show.
