Augmented Usability Tests
What to look for in a usability test for an AR app
TL;DR
We’ve conducted some usability tests on Augmented Reality apps and gathered some interesting tendencies in users behaviors here to share them with those developing an AR experience themselves.
It is often said that 97% of websites offer a bad UX and the situation is not rosier when it comes to apps, especially if they involve Augmented Reality (AR).
Lately, we had the chance to run some usability tests on AR apps and we have noticed some tendencies in user behaviors that might help you when developing your AR app.
Reality
What matters the most in AR is the perception that the user has of what he sees. As long as the illusion of reality stands, the user will enjoy the experience.
Transparency
One of the main aspects for this illusion to stand is the transparency of the technology involved.
AR adds, in fact, an extra level of complexity to the interaction with the real world: users have to see it through a mean they don’t normally use to experience it — such as their phone. This additional barrier has to be carefully dealt with when designing this kind of apps as it could make the difference between a useful and a limiting tool.
It is therefore very important that the world they see around them and the one they see through their phone feel the same. No filters, no effects, no difference.
The laws of physics
As for transparency, another aspect that can affect the perception of reality is physics. Physics governs the world around us and all of us know its functioning, much more than we think. When something does not abide the laws of physics, we feel it as unnatural.
The same thing applies with AR. If we move an object off a table and this object levitates, the illusion is broken: all we are left with is augmentation, but no reality.
Scanning the area
Reality is a very important aspect, but not the only one. The perception of space, and all processes affected by it, require us to pay particular attention when designing our app.
Space perception
Diversely to the app, which feels the space as a finite set of planes or surfaces, users think of it as infinite: they can move, turn around, even leave the room they are in.
If you place something around them, they might not be facing it, hence they could ideally walk around for hours before even noticing it at all. Hence, it is very important that you provide your users with some kind of compass to allow them to orient themselves.
How much should I scan?
Another consequence of perceiving space as infinite is the inability to feel virtual boundaries. It is very important that you provide you users with clear and immediate feedback to help them perceive how much area has been scanned, both while scanning and when trying to move past these boundaries.
What should I scan?
What we have observed, in this case, is that users tend to point the camera straight in front of them, and not at their feet, even when asked to scan the floor.
As with all human behaviours, the aim of a good design is not that of changing them, but of exploiting them to offer a frictionless experience. In this particular case, it would be better to design a scanning process that can achieve good results when the user points the camera forward, rather than forcing the user to follow a strict — unnatural — protocol.
Controls
In your app you might or might not allow your users to place objects in the space around them. If you do, you should make sure of two things: move/rotate controls should always be visible and not hidden by the object itself or anything else, and they should feel natural.
Don’t be (too) original
All domains have conventions, such as the logo in the top left corner on websites. AR does too, more than you could think. And most of these conventions are inherited from other fields, such as strategy video games.
For example, think of games as Age of Empires, Civilization or, for those of you who are younger and don’t remember these games, Clash of Clans. It is highly probable that your users are familiar with these games and the controls involved in them.
So, instead of inventing new controls, which could take time and not necessarily deliver good results, translate these conventions to the AR domain and benefit from them.
Tutorial, or not tutorial: that is the question
Ah, that is indeed a good question. In general, if your users need a tutorial to understand how your app works, then maybe your app is not that usable after all. But this does not apply to every case, of course. Some apps are not usable, but others are just complex and complexity requires a certain level of guidance to be grasped.
What we noticed, though, is that users tend to disregard detailed tutorials in favour of learn-by-doing even when it comes to new things such as AR. Our suggestion, then, is the following: if your app is very complex because the context it represents is complex (hence, such complexity cannot be avoided) do include a tutorial; otherwise, spending more time making it usable instead of adding a tutorial is by far the best choice.
Test it, test it, test it
And, as a last tip, always test your product, not only after you have developed it. You wouldn’t sail an uncharted sea without constantly checking your compass and testing is the best compass you have to check whether your project is still on the right track.
As to when you should stat testing, I’ll quote Steve Krug when he says
“start earlier than you think makes sense”.
