Cinematic Virtual Reality -a Testbed for Interactive Story


When I think of virtual reality I see interactive movies, interactive story, or whatever you like to call it. I see being able to step into a movie and interact with the characters, environment, and ultimately the story. It’s entertainment driven more by the drama than the challenge as with a game. It’s a lofty vision and one that has perplexed many.

The interactive story is a hypothetical beast in the mythology of computing, an elusive unicorn we can imagine but have yet to capture.
- Brenda Laurel “Utopian Entrepreneur

In many ways interactive story mirrors the search for the Northwest Passage, the mystical sea route through the Arctic Ocean that eluded explorers for centuries. And when it was finally traversed by Roald Amundsen in 1906 it was not the clear path James Cook and Henry Hudson envisioned but a barley navigable waterway ladened with ice. But it’s not the destination but the journey — right? Of course I hope the search for interactive story proves more fruitful and with virtual reality I see no better time to start exploring.

So where to start my journey?

Like those early explorers, I have a vision of what interactive story could be but don’t have much to go on. I’ve seen other explorers trudge out ahead of me and come up dry. Chris Crawford created Storytron but learned that his solution was too complicated. In fact what I’ve learned two things from watching the others go ahead.

  1. In the end the story has to be entertaining. Interactivity will not save a boring story. In fact many early examples of interactive story allowed users to create horrible stories. I know, I made a few of these working at Visual Purple building training simulations for the FBI and Intelligence Community. Luckily these were for training so the bar was pretty low on them being entertaining.
  2. The solution for interactive story will be more art than technology. A true artist can make you fear for Superman’s life when you know he will always come out on top at the end of the movie. It isn’t necessarily logical but artists can make you believe it. The same will be true for interactive story.

So where should I start my journey? Given that it’s a matter of artistry, should start by given them a palette to experiment — a testbed. What would artists need? But that is the wrong question to ask. Life is about constraints. The real question is — given the tech that is currently available, what would I draw upon to create a testbed for interactive story? Given the tech I’ve seen at the various VR meetups and Silicon Valley Virtual Reality Conference, what would I include in a testbed to start experimenting?

Cinematic Virtual Reality

We’ve all seen our share of demos, but the one I felt captured the visual promise of virtual reality best, is Jaunt VR. However you’ll quickly notice in the demos that the interactivity is restricted to the direction you’re looking. They tell me that they’re looking into integrating more interactivity but are just starting to explore the medium, which presents it’s own challenges. How do you have anything offset when the camera captures 360 degrees? How can you be sure the viewer saw an action when they could be looking at their feet? I look forward to see what they come up with in their first VR movie, The Mission.

Center on room exploration and lite interactivity

If the visuals are the best thing about VR at this point, room exploration is the best use of that. Looking about the rooms in Jaunt VR’s demo is mesmerizing, but all you can do at this point is look. If you could incorporate some of the simple interactivity from the exploration games I’ve seen you would have enough for basic interactive story. In many of the room exploration games you wonder about and click on items with a controler or items become highlighted when they come into your direct field of view. It isn’t much but the draw of interactive story is the drama and the best way to pull/hook people is through the characters.

A simple approach will also help when supporting multiple platforms. Mobile VR seems to be gaining momentum and while input devices are relatively undefined with the Oculus Rift, they are really undefined for mobile at the moment. And while Oculus can see a built in audience with gamers, virtual cinema might find a larger audience on mobile (at least initially).

Leverage IP to save costly exposition

The first ten pages of a screenplay are the most important as it has to get the viewer into the story. First ten pages are made all that more difficult by the need for exposition. The screenwriting guru, Robert McKee, suggests you think of exposition as ammunition, but that also takes a lot of skill. Perfecting exposition is difficult enough but given that the VR experience will be foreign for most it might suck up the majority of my resources. To not get bogged down in this I would hope to be able to pull from some IP that has a rich story world already established.

Rely on scene cuts to move the story

Stories move the most between scenes. Let’s say in one scene Superman dies. That’s quite some movement, however it will leave an even bigger question in the minds of the audience — what happens next? The leap to the next scene will almost inevitably move the story more. Also doesn’t hurt that this is probably one of the few elements from film that translates directly to cinematic VR.

What can you do with a testbed like this?

Given a testbed like this, what can one create? Let’s go through an example step outline.

IP: Inception

Concept: You just joined an Inception crew as they try to stop a ring of kidnappers. Help the crew do the job and save a kidnapped school girl.

Step Outline:

  1. First visual is a cryptic add — Long hours, tedious work, need energetic candidate. High pay.
  2. Scene opens in a warehouse with the crew’s Architect in your face. Asks you to help test out his latest construct to see if there is anything he might have missed and would alert the subject that they are asleep. Identify any object that might be out of place for some reason.
  3. Interactive project file. User can flip back and forth through it with simple arrows or more advanced UI.
  4. Reaching the end of the file opens the room exploration scene with the Architect present.
  5. User explores the room where certain objects become highlighted when in the field of view.
  6. User will select the object in some manner — with a controller, by walking up to it in scene while highlighted, using the magnet on Google Cardboard, etc.
  7. Once selected the Architect will comment. Hopefully it is funny and can also develop the story. His responses will also be part of a simple branching script as it should be responsive to which object they select first. I did many of these at Visual Purple — a lot of bang for the buck. You could also cut down on production costs by trying to shield the Architects face as he speaks. In all likelyhood the user will be searching the room rather than interacting with the Architect.
  8. Eventually the user will hit on the offending object and the Architect will thank them. He will also mention that he should get some rest while he can.
  9. The next scene opens to the user being rudely awakened by the crew leader. She needs you immediate help. The target just bailed on one of their constructs for some reason but no one knows why. She heard you were helpful before and she desperately needs your help now.
  10. And so on…

Shouldn’t there be more interactivity?

That’s exactly what I would like to experiment with — how much is necessary for users to get the impression that it’s interactive, that they have agency on the story. A research project on physically interactive story environments I came across years ago suggests that the bar might be rather low.

In particular, we found that compelling interactive narrative story systems can be perceived as highly responsive, engaging, and interactive even when the overall story has a single-path structure, in what we call a “less- choice, more-responsiveness” approach to the design of story-based interactive environments.

Leaves a lot of room for artistry. You could also experiment with branching scenes — which direction should we go? Though I’m cautious on putting questions in front of users as it can be very painful, per Mr. Payback. It should be done in a more subtle way. Branching scenes will quickly rocket up your content creation as well. If they have a sense of story agency I would consider it a success. Guess it’s like the Turning Test, if they believe they had agency on the story then you can consider it interactive story.