The Balancing Act: Creating a Helpful UI for Augmented Reality

At DAQRI, we feel that one of the main advantages of augmented reality is its ability to keep a user present in their environment while still delivering actionable information to them. If we wanted to block a person’s view, we’d call it virtual reality. We carefully craft our User Interface (UI) to ensure AR content will be visible, but not distracting. What we’re going to address here is how exactly you create a UI that provides appropriate content through knowledge transfer, but doesn’t obstruct the view of the user.

With this in mind, we spoke with Jonathan Freeman, a designer at our Los Angeles office, to gain insight into the DAQRI approach on how to design great UI for AR.

How to start from scratch

The tricky thing about designing for AR is the lack of defined UX pattern that are prevalent in mobile and web design.

You always have to start with a hypothesis or an assumption. Quickly after that, it’s all testing. Testing is crucial. You can think something’s going to work and it could be the most distracting thing imaginable. It’s all about trying something out as quickly as possible and then iterating on that. The ability to have direct user feedback from teammates helps to achieve a very quick, “hallway iterative” process.

Design in and of itself is solving a problem based on constraints. If it weren’t, then it would be free-flowing, make whatever you want. Some of those concessions made on products in general, if you were starting from tabula rasa (blank slate), would not have been made. QWERTY keyboards still exist based on a technical limitation from early typewriters. If you typed too fast, it would get locked up. They made it slower to address that constraint and now we still use that setup. Designing UI can end up being the same way. Certain concessions early on filter into the present. Iteration is the best way to move away from this. We’re not locked-in, like a company that makes a desktop platform might be.

Multi-modal interactions are where it’s going, allowing for multiple ways of input. There will be places where it’s best to have a controller, where it’s best to have a button, where it’s best to have a hands-free gaze and dwell interface. Elements of UI and contextual situations dictate the best input. As you learn more about a user base, that continues to influence UI.

When UI goes wrong

There was a photo editing app that changed their UI and ignored traditional visual language.

A person for a profile icon?

No way, we’re going to make this crazy circle thing.

It drove everyone nuts and their ratings plummeted. The lesson here is that there is a visual and functional language that you have to follow. With AR, it is still being defined as AR is relatively new without a lot of established, locked-in interaction patterns. This creates a lot of freedom.

Our goal is really well done context, avoiding things like too many interactions at the same time or having to keep absolutely still to keep something triggered. These are one way to tell that the UI hasn’t been properly tested. You have to understand the human half of the equation. Ours is centered around industrial and being functional in an industrial use case, so a lot of our UI is very easy to understand, pragmatic, and it gets out of the way when you don’t need it. Because we have more control over the space in which you work, we’re able to create a much more curated experience. As AR grows, and we can integrate it more completely with the user’s viewpoint, removing levels of abstraction, this will only serve to improve the user experience and UI.

Unlikely sources of inspiration

A huge corollary can be found between the design of AR and the design of video games. Video games already have this opportunity to show things both that are “real” and overlaid on top of the real and they’ve been playing with that for years. There are sparkly paths to guide you through a location. There are a lot of things that can be learned about placing objects into the real world from video games.

Science fiction influences design and design influences future science fiction, yet there are entire books on the disparity between AR in movies and practical application AR; what people think works and what actually works. One of the reasons behind this gap is that the AR in movies is designed to make the characters look smarter, so they have way more elements presented to them at one time than would actually be helpful in real life. The added complexity is good for the screen, but bad in practice. So while the fictional representations add familiarity, a direct export of that work wouldn’t be usable. Design must choose where to meet expectations and where it’s more useful to subvert them.

Another source of information is the use of bifocals and trifocals. This can help get a better understanding of how people actually look at and engage their world visually, a key component to developing a worker UI.

pARting wisdom

As knowledge transfer is a benefit of augmented reality, we thought it best to close with some advice on approaching UI design.

  • Always test, because it’s a necessary part of the process and you might be wrong some of the time, but start off by trusting your gut. As a pioneer in the field of AR UI development, you’re the first audience, and your impressions will feed into how the medium develops.
  • Sometimes looking cool isn’t a bad thing. Having a charming interaction, that involves a brief delay, may help them understand the interaction a little more and keep the user engaged.
  • Understand the basic principle of intent and action. Simple and beautiful is a good thing. AR is in its early stages, so you have to introduce people to it in a ease-of-approach manner. Understand that it took a lot of thought and effort to get people to understand “slide to unlock” on their phones, something that we now can take for granted.
  • One of the great things about AR is that, while it’s something that hasn’t been done before, it’s also moving back towards a more natural engagement with your world, without abstraction.

Interested in learning more about DAQRI?

Explore the way we use cutting edge technology to fuel the future today.

Connect with us on YouTube, Facebook, Twitter, Instagram and Linkedin!

And don’t forget to check out