Shit That Don’t Matter in the Alphabet Soup of VR/AR/MR/XR
We are birthing several new mediums at once – AR/VR/MR/XR. Modern culture has weathered this awkward and vulnerable stage with other new mediums: books, newspapers, magazines, radio, film and the internet that brings you this essay (in an amusingly meta turn of events, on a site named “Medium”). We can learn much from how artists and technologists navigated the beginning of film. Then as now, people initially got hung up on stuff that didn’t matter and missed the biggest new possibilities.
More than a century ago technologists like Thomas Edison were dinking around with film. He started one of the first film studios and released the first Frankenstein movie in 1910. Two groups – techies like Edison + artists and designers are the same groups pioneering immersive technology today. Here are three lessons from that formative moment in film about what shit doesn’t matter, and what does.
1. Shit That Don’t Matter: Perfectly linear storytelling
Everybody is freaking out about not being able to tell a linear story with VR, because they don’t know how to direct the viewer’s attention. The answer is, you’re missing the point of what immersive mediums are good at. Stop trying.
Interactivity
Instead, you’re going to use interactivity. Y’all from the film industry are the worst offenders. You’re trying to draw a viewer along with overly clever devices like putting them on a train and pulling them along a story line. But you don’t have viewers — you have players. Do NOT try to drag your players around by a nose ring.
Look at techniques used in gaming for best practices. They’re not even trying to tell a linear story, they’re designing for INTERACTION. They create a world, a culture, a mood and incentives for what behavior they want to encourage and then let players make of it what they will.
Little bonus: there are other places to look for inspiration on how to create interactive environments. Three fields who design compelling real world experiences are: theme parks, walking and museum tours, and immersive theater. Go forth and google.
2. Shit That Don’t Matter: Technology navel gazing
We’re going to get to that in a sec, but we have to first talk about monsters. The poster for the first Frankenstein movie had “EDISON KINETOGRAM” in huge ornate letters. Oh hey, it happens to be a story about a monster. The sentiment was “Come see this thing called a Kinetogram!” People had never seen a movie, so this was reason enough to hitch up your wagon and go into town in 1910. Then, as now, the initial draw was the novelty of the technology.
But in 1937, the Frankenstein monster you love was featured large-and-in-charge on the poster. By then, audiences weren’t coming out to wonder at the technical novelty of a motion picture, but to see a great story.
Today it’s enough to say “come see the VR demo” with scant importance on what content they’ll experience, but that moment isn’t going to last much longer. As hardware and tool development settles down, the quality of the experience is gonna be what gets the eyeballs. If you don’t have compelling content, you’re not going to get people to click download.
Obsessing over with the most perfect hardware, software and which platform to develop for should not slow you down at all from creating compelling experiences. It’s not worth it. Pick something that works for your budget and get going. It doesn’t matter which river you get in, just go get your feet wet. Hell, dunk yourself all the way in.
Shit that DOES Matter: Presence
Everybody’s talking about presence, and for good reason. We’ve never before had a medium which could so thoroughly and quickly convince our brains to remap their wiring. As builders, we are constructing worlds and new bodies for our users.
Presence is deepened in immersive worlds by several techniques, but one I’m especially excited about is its ability to enlist the whole body is the controller. You, as a builder of the experience, can target the emotions you want your player to feel via what motions you have them make. Want them to feel competitive and euphoric? Have them use a fist gesture! Want them to be calm and cooperative? Have them mirror the motions of another player. There’s more to say about that than we have time for here, but you can check out our episode on using the body as a controller and the excellent work of Jessica Outlaw and The Extended Mind.
We become our new selves first in simulacrum, through style and fashion and art, our deliberate immersions in virtual worlds. Through them we sense what it would be like to be another kind of person with other kinds of values. We rehearse new feelings and sensitivities. We imagine other ways of thinking about our world and its future.
– Brian Eno, Music Time and Long-term Thinking 1995
The more presence a medium has, the more convincingly it offers new identities for users to try on.
On console games, you *play* a character.
In immersive realities, you *are* the character.
3. Shit That Don’t Matter: What to call the medium
Early film pioneers didn’t know what to call what they were doing. Edison called his movies “kinetographs” — kineto meaning motion, and graph meaning picture. That was too fancy of a word to ever stick. “Motion picture” — that’s the old fashioned word for what we now call a movie. It’s called a movie because a movie moves and theater doesn’t. Motion was the defining new characteristic of this medium. So don’t stress about what we call this AR/VR/MR/XR thing. It will get sorted out in time and there’s not much you can do to hurry that process along.
Shit That DOES Matter: Discovering new techniques never before possible
In first years of film there was a lot of 1:1 recreation of Vaudeville theater. It took them awhile to realize the potential of the camera and editing. The biggest thing the pioneering techies and theater stars initially missed was the ability to move the camera. In Vaudeville, you can’t move the camera because there is no camera. The audience sits still in their seats watching the theater stage, so naturally the first films started with this metaphor. They placed the camera in the best seat in the house, with the frame of the film like the frame of the curtains around the stage. The ability to move the camera became the defining word for the new medium. Today we still call them “movies” because they move.
The medium is the message.
The looming question for us now is: what’s the immersive equivalent of figuring out we can move the camera? We don’t even know what these new mediums are good for. Most the content is crap because it’s making poor facsimiles of things that are optimized for other mediums. VR experiences which are a facsimile of a console game miss the whole point of what VR is for. The same goes for film. We must figure out what unique interaction these new mediums enable. That’s where it at.
With all the Shit That Doesn’t Matter — what we call this new medium, which hardware and tools to use, or trying to drag your player through a perfect linear story line — don’t let that get in the way of the Shit That DOES Matter. Get started and try lots of things. The watchword is action. Create experiences that take full advantage of presence and interactivity. Get curious in the real world about what makes environments and interactions compelling.
What monsters will we make?
The last parallel I’ll draw is of what the Frankenstein story was about — what hazards does new technology bring us? Dr. Frankenstein experimented with the new tech of the time, electricity, and ended up with a monster that killed his wife and destroyed his life. Hey, don’t blame me. I didn’t write it. The story was about the technology and it was told with the technology — an electric light bulb and movie projector. It’s so meta it hurts.
I’m starting an org like Code for America, but for advancing technology with science
We are unintentionally making technological monsters today. If you think your smartphone is addictive… immersive tech has the potential to be even worse. To actually DO something about it, I’m assembling an organization, Hack Reality, to do foundational research on emerging immersive technologies — AR/VR/MR/XR plus biosensors and more to investigate how to do no harm with these new mediums. It will connect developers and designers with neuroscientists and cognitive psychologists to build non-addictive tech.
We’d love to have you help us create this movement together.
You can find my interviews with emerging tech creators and more on the Embodied Reality podcast on iTunes and everywhere podcasts are found – plus videos on the YouTube channel. Please leave a comment and tell me what content you love that uses these principles.
Thanks to my co-conspirator, Tim Riot. His editing is the reason you’ve read all the way to the end. ❤
