Designing intuitive UI/UX for AR
Designing AR experiences that are intuitive and easy to use is a crucial element of AR content creation. Yet it’s something that can be very easily neglected by solo or small team creators — especially when juggling multiple roles without formal user experience training or the benefit of a QA team. But the fact remains that while the normalization of AR continues apace, the need to on-board and empower curious users is more important than ever. After all, if users don’t know how to use your AR experience, they move on and miss out on all the fun. Nobody wants that, right?
UI (User Interface)/UX (User eXperience) design addresses this incredibly important part of crafting memorable AR. It is the gateway to your carefully curated experience — the method that informs how users navigate, interact and ultimately learn how to get the maximum benefit from their time with it.
Helping users get the most out of AR is a huge focus for Zappar and the wide range of brands we work with and we felt it was important to share our experiences with the ZapWorks community. That’s why we’ve drawn together our internal expertise to find out what it takes to create rewarding AR experiences that your users will love.
Defining UI/UX
UX is a broad field that “encompasses all aspects of the end-user’s interaction with the company, its services, and its products”, to borrow a phrase from Don Norman, the man who gave birth to the definition. The discipline has become more nuanced since its inception but at its core, it’s approaching product design from a human-centric perspective — it is the method that enables users to get the optimum benefit from a product by making their experience intuitive and ultimately rewarding. This means it can cross-over into a lot of different disciplines — from market research (exploring how users are likely to discover and interact with your product), to design (optimizing your product with those users in mind) to project management (tracking user interactions, analysing them and iterating upon them).
Meanwhile UI, in the absolute simplest terms, can be thought of as the group of buttons and prompts that the user interacts with in an experience…while being so much more than that. Think of it as a bridge that connects the user with the experience itself — it’s how the user learns how to interact with an experience, and therefore incorporates the graphic design of interactive features such as buttons or gesture/movement and sound prompts.
So while UI and UX are often mentioned in tandem, UI design is actually a facet of UX design. Fantastic UI design is informed by the overall UX approach. As Rahul Varshney, Co-creator of Foster.fm puts it:
“A UI without UX is like a painter slapping paint onto canvas without thought; while UX without UI is like the frame of a sculpture with no paper mache on it. A great product experience starts with UX followed by UI. Both are essential for the product’s success.”
Why UI/UX design for AR is different
The fundamental principles of user-centric design persist for AR, but the medium does bring specific challenges to adapt to.
For example, with mobile game design the UI is predominantly on-screen, but when we talk about AR, the UI can be embedded in the ambient in a layered form because of the integration of 3D real world space into the experience. AR enables the use of ‘floating’ buttons that exist as objects in this 3D space that users can intuitively follow, rather than static on-screen button prompts which can have the effect of taking users ‘out’ of the 3D experience that you’re curating.
“The user’s portable device is the window into the enhanced world. This portability means that a designer must always consider the user’s environment because it’s open to so many variables, which in turn dictates and effects the UI placement, shape, size and colour.”
- Joe Brazao, Lead UX Designer, Zappar
Another layer is that unless the AR experience is only being triggered in a very defined space — like at an event — you have to consider the context of a real world background. Thinking carefully and testing opacity, color and shadow is crucial to maximise visibility in a wide range of settings is really important. Remember, a user could be scanning while sat on the bus or in their front room…almost anywhere!
World tracking and face tracking — UI opportunities and challenges
Advances in AR technology lead to new functionality, which in-turn brings an increasingly wide range of ways for users to enjoy experiences. But while this presents a fantastic opportunity for innovation and creating ever impressive AR, it also brings a significant set of new challenges to the table when it comes to UX/UI design.
World tracking, a headline feature of Apple’s ARKit which will soon be available for ZapWorks Studio, means some truly significant changes in our expectations of user behaviour. For example, content can now be attached to flat surfaces in the real world, which means a broader set of possibilities compared to experiences which are tied to a single tracking image.
Also, objects in AR can now remain in a specific place off-screen while a user explores a different part of that space. So there’s a need to explore ways of introducing new UI that adapts to the real world, which in-turn brings the challenge of sensitively encouraging users to either return to or seek out new content as they explore. For example, actively prompting users becomes more important as the functionality will often be unfamiliar, such as teaching them to tap on the screen to place content in the new world.
Actively prototyping projects, testing and close collaboration between developers and UX/UI designers becomes even more important for our team — building testing into an iterative design process focused on assessing any user experience issues that arise to achieve the best possible outcome. After all, it’s easy to see the success of products such as Pokemon Go and make the assumption that users are now going to react to world tracking experiences intuitively — but this is still very new to lots of people.
“I worked really closely with the developers and it’s through that collaboration that I began to see the wider applications of AR, and therefore the range of both UX challenges and opportunities that come from the introduction of world tracking. From a design perspective, I can see examples that I think look cool but in the end, it might not be appropriate from a user experience perspective. So the key is collaborating frequently with the team and letting colleagues from a range of disciplines, stakeholders and users (when appropriate) experiment with the prototypes on a regular basis.”
- Filippo Marchiafava, UI/UX Designer, Zappar
But while new functionality brings new challenges, it should fundamentally be exciting! The more features that evolve in the AR space, the more possibilities that unlock for creators to explore. But if you’re an AR creator exploring this landscape, how do you integrate these UI/UX principles into your existing methods?
Integrating UI/UX into the AR creation process
Our team of QA (Quality Assurance) testers provide additional feedback before any experience is released, to make sure it works as proposed and is easily understandable. This is also a great opportunity to include colleagues who are not familiar with the project — something we call ‘guerilla testing’. It’s really crucial to have ‘fresh’ eyes to help out when assessing your user experience. When you spend so much time testing a project, your familiarity with it can cloud your judgement so it’s important to keep that in check by observing how different people interact with your prototypes.
Back to basics with pen & paper prototyping
Despite AR existing resolutely in the digital space, storyboarding is a really helpful UX method. By visualising the user journey, you can imagine the user approaching the experience and the context they are in. Are they at home? Are they at the supermarket? Is it an experience we think they’re going to unlock at a specific time of day? If so, what would the lighting likely be like? Is the user aware of the best possible way to experience the AR project?
Our work with Gallo Wines is a particularly good example of implementing this process, where the UI sensitively and quickly lets the user know where and how to get the best out of the experience in an unobtrusive way. Don’t be afraid to utilise guiding copy — ease of use is the priority.
All of these different settings have to be considered when designing the UI and storyboarding really helps focus on that. To quote the experts over on UX Design World;
“…if a picture is worth 1000 words, a prototype is worth 1000 meetings!”
It’s so much more cost and time-efficient to identify potential UI issues at this early stage rather than having to re-design parts of an AR project further down the line.
Rapid prototyping and the import pipeline with ZapWorks Studio
The ability to prototype quickly in ZapWorks Studio is a huge benefit for user testing. Some software companies are beginning to listen to demand for testing AR in a real world context but predominantly, your testing is limited to basic sliding and tapping on your device. It’s just not enough.
“When it comes to creating UI for AR, you need to see the experience as it develops in a full 3D space — in front of you and in different contexts — which is why Studio’s instant preview functionality is so important.”
- Ayesha Evans, Senior QA Tester, Zappar
The import pipeline for our internal team is fairly simple and can be adapted depending on the tools creators prefer to use. Here’s UI expert, Filippo, with his preferred method:
“The process usually starts with initial concepts and storyboarding before moving into Adobe XD, which I use to explore the visual style of the project. This includes how interactive buttons could fit sensitively into the overall aesthetic and begin to think in terms of user flow. Mood boards can help with this process. By gathering imagery with similar themes to the project you’re working on means you can see how the colours interact and gain inspiration.”
- Filippo Marchiafava, UI/UX Designer, Zappar
Remember that the use of colour can provoke a different emotional response from users from different backgrounds according to social norms. For example, will your users perceive a red button as positive gateway, or does it imply stopping? If there are projects already out there with similar goals to your own, make sure to experience them — how could you iterate upon their UI/UX choices? Would you make changes for the demographic you were targeting?
One of the big strengths of ZapWorks Studio from a UI/UX perspective of iterative design is that it’s fantastic for testing on the go. If you want to test a build of a mobile game made in Unity, you have to put that on to the phone you wish to test on, and any additional phone too. Whereas with ZapWorks, you can just create a code, scan and go.
AR for everybody — utilising toolkits and accessibility
Our internal UX/UI professionals utilise a wide range of software tools on their AR journey — such as Sketch (particularly for prototyping), Photoshop and Illustrator for more complex visuals. Likewise, SketchFab is a really useful resource for UI/UX designers, when you need assets quickly for prototyping — functionality that we are excited to say is being integrated into ZapWorks Studio 6.
Many of these toolkits are absolutely vital when pursuing a broader range of accessibility options for our experiences — so when considering the UI/UX of your AR experience, try and ensure as many people as possible are going to be able to enjoy it. Optimising your experience to make it usable for people with colorblindness, through understanding colour patterns and learning to add texture to images, is one great way to make your AR experiences more accessible.
“One thing I’ve been passionate about exploring is accessibility. I’ve been testing prototypes to see how appropriate they are for people who are colourblind, which I think is really important…It’s a great way to make things easier for people so that as many people as possible can enjoy AR.”
- Filippo Marchiafava, UI/UX Designer, Zappar
Examples: Creating AR experiences for brands that are user-centric
So, enabling the user to interact with experiences in a natural and intuitive way is the primary goal, but creating content for brands comes with an added layer of responsibility. Our HQ in-house creative team are well aware, having developed AR digital marketing campaigns for iconic worldwide brands such as PEZ, Rovio and Walmart that have been scanned by thousands of people. But what these campaigns share are the inclusion of iconic mascots that hold real emotional weight and have a high value to a brand’s identity — so a poor user experience reflects poorly on clients.
“It’s about acknowledging that if an AR experience is going to carry a brand’s name, then they need their customers to spend quality time interacting with it in a way that’s both easy to use and compelling. It’s a direct connection between the brand and their audience, so it reflects directly on them. That’s a high level of responsibility from a UI design perspective.”
- Filippo Marchiafava, UI/UX Designer, Zappar
Our work with 7-Eleven is particularly well loved internally — it’s a really intuitive set of experiences to pick up with a natural scroll function. It’s colour-coded specifically for the environment of the stores too and we particularly like the integration of sparkly animations to the buttons, provoking an emotional response in the user and letting them feel rewarded for unlocking content.
Animated buttons are a ZapWorks feature that you can leverage to communicate with your users in an efficient, rewarding way. They’re a key part of our Ancient History experience tutorial — why not explore implementing it into your next project?
Our Senior QA Tester, Ayesha, is particularly passionate about the user-centric design of our festive range of AR experiences created for Walmart and their brand partners. These utilised the power of ARKit which meant some fantastic possibilities — but with the experience appearing in a 1000 stores across North America, it brought a range of UX challenges due to the potential of users being less familiar with the relatively new technology. Internal cross-team feedback was pivotal in addressing this, particularly with the crucial onboarding animation which went through multiple iterations. But with good communication and rigorous testing, this iterative design process delivered a really intuitive user experience.
“In terms of multiple user journeys, it combined a more familiar menu flow for a wider audience and the slick scan to experience user flow.”
- Ayesha Evans, Senior QA Tester, Zappar
Meanwhile, it’s our Carrefour x Minions AR campaign that captured the imagination of QA Tester, James. This experience was all about collectability — encouraging users to track down limited Minions-themed cards. This meant getting people excited, immersed and feeling rewarded for participating was at the heart of the user-centric design process. Animating treasure chests in combination with grand audio cues gave users a real sense of occasion for unlocking content, with the interactive inventory giving a subtle call-to-action by highlighting gaps that were just waiting to be filled — in other words, user-centric design that keeps the user focused upon a goal, to increase engagement. Meanwhile, the UI featured straightforward iconography — with buttons that were immediately easy to understand to compliment quick tutorial pop-ups that were explanatory for beginners while being easily skippable so as not to frustrate longer term players.
“I’d say with Minions, having to collect them all to unlock all the content and complete the set is really fun and is designed with rewarding users at the forefront. It incentivises the user to collect more cards and unlock more content with each scan”
- James Stewart, QA Tester, Zappar
Top tips: avoiding common UI/UX pitfalls in AR design
A lot of pitfalls can be avoided by giving proper consideration to the context the user is likely to be in when loading the experience. Will they have a stable Wi-Fi connection? How much time are you expecting them to spend with your experience? Does the scanning context enable that?
To ensure that the user understands the gist of the experience they are about to engage with, giving the user a strong on-boarding guidance is crucial. It’s important to remember that even if you are an AR creator and you’re familiar with the medium, it’s still a very new technology for a lot of users and they might not instinctively know how to interact with it.
For example, some people may scan an experience and then walk away when they’re supposed to stay with a tracking image, which is why ‘look prompt’ functionality is so useful. So having a strong call to action is really important. The user needs to quickly know what is going to happen if they scan a code and where that experience is going to happen (i.e. in front of a tracking image or in a particular space).
These user experience challenges will diminish alongside the normalisation of AR in everyday life — many people already know how to interact with a face filter experience, for example, so a lot of traditional icons that prompt users to zoom in, rotate etc. have now been removed. The more people interact with AR and get familiar with it, the less time is required to spend assisting people early-on in the experience. But for now, explicit onboarding is crucial for users to know how to interact with an experience.
“Using AR for the first time is thrilling, but also brings a user into an ‘alien’ environment. So, how can we help a user to understand the value of such an unfamiliar experience from the get-go? It often raises some questions — ‘How do I interact with this?’ ‘What comes next?’ There is still an obvious learning curve for users when it comes to AR which needs to be addressed in our approach.”
- Joe Brazao, Lead UX Designer, Zappar
One very important aspect of this is the scaling of the UI. With on-screen UI, a user is always going to see the options you’ve presented them with, if you have followed standard design rules. But with AR experiences, there is a variable distance in the digital space between the user and the tracking image. For example, when a user scans an experience, the content’s size can vary depending on how far or close the user is to the tracking image. So it’s important to test scanning in a wide array of settings — making sure that your initial UI instructions are scalable to be read regardless of where the user is.
At its core, a lot of UI challenges for AR come down to the UI and UX best practice (Laws of UX is a fantastic resource, for example) you would see in mobile gaming or indeed any other app experience. To use a generic example, if you want the user to tap a button at a particular time or indicate that tapping will trigger a reward, then draw their attention by animating it — it’s the same principle in terms of prompting a user reaction in AR.
However the penalties for poor design choices in AR are arguably higher than in other mediums. Unless the user is provided with an attractive incentive to scan and engage with the experience, a UI that’s unintuitive will ultimately end in AR project disaster and user frustration. A user may persist with a downloaded mobile game as the value proposition is more obvious & immediate — but with a free AR experience, if you don’t make it easy to use, users will walk.
Pen and paper prototyping is low tech but a really great, methodical way to start thinking in terms of how a user will experience what you have created. By holding the paper up you can imagine the AR ‘box’ floating in the air — how will the user interact with it? What do they need to tap? This can then inform the storyboarding process and what journey you want the user to take, from start to finish. Mapping out the experience on paper before you begin creating it can identify potential issues early on, before investing lots of time in the actual development process unnecessarily.
“…this is getting granular, but it’s really important to leave about four pixels of alpha margin around your buttons for the UI when exporting! Forgetting this is a common user pitfall, so it’s important to remember.”
- Filippo Marchiafava, UI/UX Designer, Zappar
Troubleshooting UI/UX with AR
When it comes to testing your own AR experience, you’ve got to learn to be impatient. Test with the assumption that the user has not got a significant amount of time to sit and experiment.
How long does the experience take to load? Is there a lot of interaction required before the experience begins in earnest? If you’re a creator without a dev team to bounce ideas off and carry out internal testing with multiple people, you can always let your friends or family try out the experiences. Try to get an informal cross section of people both familiar and unfamiliar with the kind of experience you are creating — observe how they interact with it, note their feedback and implement the findings.
“Even as a UI designer, I know that when I open most games or AR experiences, I skip the tutorial text and pop-ups. So I make sure to test the experiences we create in the same way, building a degree of empathy with a wide range of different potential audiences.”
- Filippo Marchiafava, UI/UX Designer, Zappar
AR is not like a long-form desktop experience such as a PC video game, where the assumption is logically made that the user with a keyboard and mouse has time to dedicate to exploring the experience. On a smartphone, users tend to do everything with one hand and different individuals have very different assumptions about how to tilt the phone screen. That’s why it’s great to implement an icon that explicitly tells the user how to orientate their device. It’s far less problematic to either notify the user of the proper orientation to begin with or to make sure the UX/UI works without issue in both portrait and landscape mode.
Ambiguity causes more problems than an additional line of copy or an extra button — so don’t make assumptions about user behaviour and make sure you test in-app from a wide range of perspectives. Filippo recommends the Space Ape’s series of talks as a useful way to learn about user familiarity in this context. It’s an important balancing act between the desire to be innovative in user controls, for example, with not confusing users by taking them too far away from the behaviours they find intuitive.
“For me, design has to be about functionality first and aesthetics second.”
- Filippo Marchiafava, UI/UX Designer, Zappar
ZapWorks Studio’s preview tool means that this can be tested in-app immediately without saving a build, so it’s a really quick and easy element to test. It certainly has a tangible impact in terms of saving both time and money with key client relationship benefits too when it comes to working with brands. Instead of lengthy waits for new builds, client feedback can be implemented quickly into projects, enabling you to send across a Zapcode instantly for them to see the changes.
Lightning Round! Our top tips for user-centered interface design for AR
1 — Don’t assume familiarity with AR. Guiding curious users who are less familiar with the technology is crucial for mass engagement.
2 — Animation is emotional. Add some interactive sparkle to your buttons to guide your user sensitively and make them feel rewarded for joining in.
3 — Get out the pen and paper. Storyboarding and laying out your user journeys physically is a cheaper and more efficient way to spot hurdles than having to re-design your experience.
4 — Prototype early and often. If you’ve got your core user journey in place, share preview builds with a client early. The earlier the feedback, the easier it is to implement.
5 — New technology means new UI. World and face tracking bring both new opportunities and challenges for UI designers to consider.
6 — Be impatient! Respect your users’ time — how does your design stand-up when you skip tutorial or instructional content?
7 — Friends and family are the QA team you never had. Let the people closest to you loose on your creations and get some honest, valuable (and free!) feedback.
8 — Context is king. Design with a wide range of environmental settings in mind. Where and how will your users be engaging with your experience?
9 — AR creativity means community. Online groups and forums (such as the ZapWorks Forum) are a fantastic resource to share ideas, solicit feedback, informal QA and to learn from others’ experiences.
10 — Research plug-ins and techniques that enhance accessibility — AR is best when it can be enjoyed by as many people as possible.
Final Thoughts
One of the most important lessons about creating fantastic experiences for AR projects is getting fresh perspectives from a wide range of potential users. So if you’re a creator, why not visit our friendly group of passionate AR enthusiasts over on our ZapWorks Forum? The ‘Show and Tell’ section is a great place to share your work in progress and get valuable feedback from people from all over the world. Remember, world tracking and face tracking are coming soon to our ZapWorks Studio toolkit which will be available at the end of February. If our exploration of UI for this new era of AR creativity as got your imagination flowing, then why not get in touch and see how we can make your vision a reality? We can’t wait to see what you come up with!
Originally published at www.zappar.com.
