Thoughts on VR Design

Hey I’m Rafael Kino and I’m currently looking for a position in Film/Design/Games in the UK. Check out my portfolio here.

A few months ago I wrote a few sentences about why and how I think VR and AR could change everything about the software we use. People kept asking me about those paragraphs, so I’m writing a new article. Hopefully this will clear some doubts on why I’m so excited about the future of Virtual Reality software.

A 3D World

I believe that in order to think about VR Design, we have to reboot Software Design. “Flat” Software design such as the one we have today in our software is different from Volumetric or 3D Design. Flat Design is inspired by Graphic and Industrial Design. VR/AR design is more of a fusion between Architecture, Sculpture and Animation. Volumetric software comes with different principles and constraints than flat software, and it should be thought of and implemented differently.

There are multiple limitations in 2D interfaces that we’ll surpass by using Volumetric interfaces:

  • 2D Graphical User Interfaces render everything at one distance from the user in a flat screen.

Designers and engineers have worked around this for ages by using depth cues, like shadows and transparency. In applications where you have to work in 3D, like 3DsMax or Blender, we get multiple perspectives, oftentimes 4 camera views all rendering at once, just so we can understand 3D! That’s not necessary in VR/AR, because you already have stereoscopic vision, so you see in 3D.

  • The viewport for our computers still looks and feels a lot like “Digital Paper”.

2D GUIs are still centered around the metaphor of papers and folders stacked on a “desktop”. 2D GUI Design today is still so indebted to the work done at PARC in the 70s that Ted Nelson calls it PUI for PARC User Interface. This is a chance to start anew and wonder: what do we want computing to be? Can we imagine it differently?

Add to this the growth of Natural Input Interfaces using voice, eye and hand tracking and we have the basis of a revolution in Human-Computer Interfaces. This is not pure speculation, take a quick look at Eyefluence a startup that thought of using eyes as a form of input for AR headsets. Here’s a demo of their interface.

What could a VR/AR world look like?

First, I have to say I have no idea what the world will look like in 10 or 20 years. We can only imagine and speculate.

One way to focus our imagination (and feed it with data) is by reading the immense amounts of research that’s already been done on VR and 3D UI/UX, starting with Doug Bowman’s excellent “3D User Interfaces: Theory and Practice”. Bowman has tons of incredible information about how Designers have worked out new ways to do things in 3D, what works and what doesn’t, and what cognitive science tells us about human thinking and how we can use that to design better UI. I highly recommend the purchase.

Now if I was to choose a possible future, I would base it on one particular paper that set fire to my imagination - “The Role of Social Presence in Interactive Agent-Based Persuasion”.

A bit of background — one year ago I was working at a VR company and they wanted to sell VR to large corporations for training. They felt like if we could find scientific evidence for benefits of training in VR, we could make a stronger case. I read quite a few papers on the subject, but the evidence was contradictory. We couldn’t really prove that VR made for better memory recall or persuasion. With exception of one particular paper, which did find a positive connection.

Here’s an excerpt:

“Results of causal model tests suggest that interactive agents facilitate social presence leading to increased message processing, which in turn affects both attitude and behavioral intentions (…).”

By interactive agents they mean characters. They were testing whether using characters in a VR scene meant you would be more open to persuasion by the message of the experience. The key to the effect, and the authors discuss this in more detail in the paper, is what they call the Heuristic Systematic Model. The model predicts that the more systems you engage in your subject, the likelier you are of your message being received. The paper discusses that vividness and interactivity are the origin for “social presence”, and social presence is the key for better message processing in users.

Now this paper blew me away. What can we imagine from this if we open our mind’s eye?

“Who Framed Roger Rabbit?”

I’m fascinated by this film. If characters, or social agents, are the best way to make users interact with VR/AR software, and users have portable VR/AR headsets, could the World look like that?

I love the idea that one day software would be characters. Software programs could walk among us and have a personality. Your credit card could be a character. Amazon could be a character. Thinking about this is most fun I’ve had imagining a VR/AR world. Brands could finally have full expressions of their personalities as characters and you would interact with them everyday.

Imagine this…

It’s 2022 and you own a pair of AR glasses. They can project “photons into your eyes”, so 3D objects look real. For their new version of Hololens, Microsoft builds “Microsoft Accountant”.

When you wear your AR Glasses the Accountant character:

  • Tells you how much you’ve spent on different goods this month
  • Tells you much you have left to pay on your mortgage
  • Keeps doing quick searches for better deals on your investments
  • Gives you quick nod for “Yes” or “No” whenever you’re at a shop holding an item or getting close to checkout, telling you whether you’re spending more than your budget for this month

I think on some version of a Volumetric future is both cool and inevitable, so we might as well be on the driver’s seat. It’s a myth that it doesn’t matter who or how we build our futures, that it’s inevitable, or that everyone will reach the exact same conclusion. I am a big believer in the influence of individuals to shape their future. With my background in Animation, Film and Storytelling, I look at VR UX and see the biggest opportunity of my lifetime. Mobile Design has taught us that Animation matters, that users care, and that they want to be soothed and awed by their technology.

Amazing futures like this one are imaginative and fanciful, yes, but they’re a wonderful way to spend the time (it’s even better to build them!). I can see incredible possibilites in VR Design, the possibility of changing computing for the better, to make it more poetic and humane. When I look at our VR/AR/Holographic future, I think what a wonderful world that could be, if only it had more beauty (and less datamining) in it, and I’ve decided to take my career in a direction where I can shape it.

Rafael Kino 2017