VR and AR Design Idea #1 — Software as Characters

An Introduction

What if VR & AR become a major way we interact with computers in the near future?

One of the reasons working in VR & AR is so exciting is because if these technologies become commonplace, we will (hopefully!) do a reboot on Human-Computer Interaction Design. 3D software comes with different principles and constraints than flat software, and that means there is a chance to think and implement it differently. Just off the top of my head, there are multiple bad things about 2D interfaces that we’ll immediately get rid of by using Volumetric or 3D interfaces:

  • 2D Graphical User Interfaces render everything at one distance from the user in a flat screen. This limits their potential in a lot of ways, one of the more obvious one being 3D work, where you often have 4 camera views to understand what you’re doing.
  • And the viewport for our computers still looks and feels a lot like “Digital Paper”.

Most of the work around the desktop metaphor comes from the work done at Xeroc Parc in the 1970s, where the intention was to build “the office of the future”. GUIs are still very similar to the interfaces done at PARC. This is a 40 year old software paradigm, designed for a time and place that no longer exists.

At the same time as VR & AR are growing into viable tech, we are also seeing the explosive growth of Natural Input Interfaces - using voice, eye and hand tracking to communicate with our computers. Combining VR/AR with Natural Input Interfaces we have the foundation for a revolution in Human-Computer Interface Design. Think of how smart and comprehensive Siri and Google Assistant have become, and take a quick look at Eyefluence - a startup that figured out how to use your eyes as a form of input for AR headsets.

What could a VR/AR world look like?

Predictions of any kind have a way of being mostly wrong, so I’ll start by saying I have no idea what the world will look like in 10 years. I can only speculate. One way to focus the imagination when thinking about VR Design is reading the immense amounts of research that’s already been done on VR UX. And if I was to choose a possible future I would base it on one particular paper that I’ve grown to love - “The Role of Social Presence in Interactive Agent-Based Persuasion”.

Here’s an excerpt:

“Results of causal model tests suggest that interactive agents facilitate social presence leading to increased message processing, which in turn affects both attitude and behavioral intentions (…).”

By interactive agents they mean characters. They were testing whether using characters in a VR scene meant you would be more open to persuasion by an experience. The key to the effect, and the authors discuss this in more detail in the paper, is the Heuristic Systematic Model. The model predicts that the more systems you engage in your subject, the likelier you are of your message being received. The paper discusses that vividness and interactivity are the origin for “social presence”, and social presence is the key for better message processing in users.

Now this paper blew me away.

“Who Framed Roger Rabbit?”

I’m fascinated by this film. If characters, or social agents, are the best way to make users interact with VR/AR software, and users have portable VR/AR headsets, could the World in 20 years look like Roger Rabbit? Could we be looking at a future where animated, synthetic software walks among us, existing only in the “Metaverse” of the simulated software running in our devices?

Your credit card company could be a character. Amazon could be a character. Imagining a world where software are characers is the is most fun I’ve had imagining a future VR/AR world.

Imagine this…

It’s 2020 and you’ve just bought your first pair of AR glasses. They have wide FOV can project “photons into your eyes”, so 3D objects look real. Microsoft has just released “Microsoft Accountant”.

When you wear your AR Glasses the Accountant character:

  • Tells you how much you’ve spent on different goods this month
  • Tells you much you have left to pay on your mortgage
  • Keeps doing quick searches for better deals on your investments
  • Gives you quick nod for “Yes” or “No” whenever you’re at a shop holding an item or getting close to checkout

In Amazing futures like this one are imaginative and fanciful, yes, but they’re a wonderful way to spend the time (it’s even better to build them!). I can see incredible possibilites in VR Design, the possibility of changing computing for the better, to make it more poetic and humane. When I look at our VR/AR/Holographic future, I think what a wonderful world that could be, if only it had more beauty (and less datamining) in it, and I’ve decided to take my career in a direction where I can shape it.

Rafael Kino 2017