2 Books You Must Read (Again) Before You Start Designing for AR & VR

There is no shortage of books on design, but when designing for AR & VR, mediums that are at the early stages of reaching mass markets, it’s hard to know where to start. The three books I will discuss below help inspire designing for this medium; from understanding interaction patterns, to building a strong creative environment, to imagining the endless bounds of what is possible.

Designing for AR & VR requires you to step away from the way we’ve been approaching interface design for the past several decades, and start thinking in the 3rd dimension. As bold as that may sound, the introduction of Z-space and dimensionality forces you to challenge all the patterns and assumptions of 2D design that you’ve been operating within.

• How do I ensure the user looks at the thing I want them to focus on?
• What happens when the user looks at this element from behind, from above, from the bottom?
• How do I ensure a user feels grounded in an unfamiliar interaction pattern?

While interacting with immersive (3D/AR/VR) content might be very new to many people, the patterns are actually right in front of our nose — the secret to these patterns are in how we interact with physical objects in the world.

Which brings me to the first book…

The Design of Everyday Things by Don Norman

This book is foundational for user experience design, but even more so for designing for AR & VR. The qualities of physical objects and the things that make them usable are the same principles we must consider when designing virtual things meant to feel like they’re in the real world. When designing immersive experiences, an understanding of affordances, signifiers, and symbols is critical to help users understand how they navigate your experience . This book breaks down those affordances and how to best use them in the context of real objects we come across in our lives every day.

The first time I tried the VR experience Job Simulator, I was completely blown away. While the visual style was cartoon-like, I felt like I was actually in the experience because I interacted with things just like I would in the real world. Doors had handles on them that you pulled, buttons could be pressed, objects could be picked up and set back down. Qualities that mirror those of the world we are familiar with help create presence in virtual experiences, the sensation that you are actually there.

When designing new experiences, it is paramount that they are usable — and The Design of Everyday Things is a crash course by one of the finest thinkers in this area.

Designing new experiences also requires a strong team culture built on trust and collaboration, which brings me to book #2…

Creativity Inc. by Ed Catmull

Creativity Inc. is not only a user manual for putting together a team to accomplish seemingly impossible goals, it is also a history of the 3D graphics industry.

Creative principles in animation are very similar to those of creative characters and objects created for augmented and virtual reality.

The definition of superb animation is that each character on the screen makes you believe it is a thinking being. Whether it’s a T-Rex or a slinky dog or a desk lamp, if viewers sense not just movement but intention — or, put another way, emotion — then the animator has done his or her job. It’s not just lines on paper anymore; it’s a living, feeling entity. (Excerpt from Creativity Inc.)

We look at character design in augmented reality through a very similar lens. The more integrated objects are in the natural environment, the more compelling they become. When designing characters, the animation and behaviors are what brings them to life, taking something that feels like a flat object super-imposed onto a photo or video, and transforming it into a physical object integrated in the world.

Working in 3D design, I found this book to be so inspiring because it is the origin story for a lot of this type of content, and acknowledges how challenging it was (and continues to be) to work in this space. It pairs so well with Don Norman’s book because once you identify what you want to create, you need to figure out how you are going to do it — and you need a team culture set up in such a way that fosters creativity and innovation, while being resilient enough to get you through turbulent times.