The Laws of Simplicity Applied to VR

Paulo Melchiori
3 min readJan 2, 2022

--

6. Context

What lies in the periphery of simplicity is definitely not peripheral.

Illustrated by Arthur Petrillo

Augmented reality and mixed reality can give you superpowers. But virtual interfaces have to navigate a thin line between being useful and annoying to users and those around them.

Imagine Tony Stark in his suit fighting Iron Monger in the first Iron Man movie — by far my favorite. Tony and Jarvis, his AI assistant, working as a team, Jarvis making proactive suggestions and providing all sorts of combat information through sound and virtual display seamlessly, ultimately leading Tony to pin down Obadiah semi-unconscious inside his massive suit. As Ironman is about to serve his final deadly strike, Jarvis suggests a few options on the screen:

How would you like to end this fight, sir?

a. Flying punch to the head.
b. Chest-armor power shot.
c. Ultra-voltage electric shock.

Tony hesitates for a few seconds considering his options. Obadiah catches a second wind and quickly initiates his back missile launcher. He aims it at Tony, who can’t see it as the virtual interface is covering part of his field of view. Obadiah pulls the trigger. Iron Man dies. The end.

Understanding the context in which an interface is displayed is always important, but it is perhaps most important in AR / MR where interfaces can quite literally get in the way of what you are doing. AR interfaces will give us superpowers like knowing directions, comparing prices, giving extra information about people and places, pointing out hazards and how to avoid them. But to be helpful, these interfaces will need to be minimal and aware of their context. They need to provide maximum value to justify their appearance, while occupying the minimum field of view to not be distracting. And they need to be aware of the physical environment you are in so they can display only relevant content, preferably anchored to objects, places and people, instead of tethered to the user’s vision.

Understanding context is also important for designing inputs. While hand gestures and voice commands will certainly continue to play a role in how we interact with virtual interfaces, we’ll need to create discrete inputs that are more socially accepted and don’t disturb people around you. This will likely mean some of the interactions will require using a second device, such as your phone or watch, as your AR input. You don’t want to be that guy gesturing by yourself, or voicing commands out loud across the room. That is assuming that you don’t walk around in a red and gold suit. If that is the case, go ahead, more power to you.

--

--

Paulo Melchiori

Design leader for emerging technologies. UX Design Director at Google AI, Gemini. Former Alexa (Amazon), Oculus VR (Meta).