Adapting Standard XD Practices to Handle Unknowns in AR (And Beyond)
For experience designers and UX-adjacent creatives, an AR/VR project has all the things to make the work meaningful and fun. (read: challenging) A perfect marriage of innovation, applied design and experience, and often no small amount of unknowns to keep you on your toes. Equal parts exciting and daunting, taking on an augmented reality project often involves seeking out some expert advice. Be it template recommendations, workflow considerations, or simply ‘Where do I even start?’ As luck would have it, I was in this exact position not too long ago and I have answers. Some, anyway.
Here is where this article becomes half a personal experience play by play and half tips and tricks for your brain to digest. The AR/VR project I was part of was an 8-week internal exploratory initiative (a ‘lab’ as the startups like to call it) we were piloting at Slalom Build. Our cross-functional project team of 5 didn’t have much spelled out in terms of deliverables or direction — the only concrete ask was to document as much as possible and share our learnings across the company, highlighting our individual capabilities’ point of view.
POV = Point of Variance
The first iteration of this lab was tasked with exploring Azure spatial anchors. This new, still-in-active-development technology brings spatial awareness to an AR or mixed media experience — be it multiple users sharing an AR environment, multiple holographic points interacting with one another, or content that persists in the real world. It’s AR upgraded!
The first week or so was a blur. All capabilities (engineers, SO, QE) jumped head first into fast and furious explorations, working side by side in uncharted territory, getting up to speed with AR, and simultaneously exploring spatial anchors and the limitless possibilities they offer.
About halfway through the project it become clear that I was heading down a bit of a different track than my lab mates. For the engineers and QE team members, the tech was so new (meaning little documentation) and different (Unity/mobile development) that the majority of my time was spent ramping up to a point where spatial anchors could be the focus. I was doing my best to fill in the gaps, including some light front end design where it could be easily applied, but it was clear there was no need to produce anything resembling a product or fleshed out experience.
While I faced a bit of an existential designer crisis, I was determined to produce something that could stand alongside the engineers’ documentation. After a few pivots and quite a bit of heavy grumbling I started to shift my entire thought process from “How do I design for AR?” to “How can we think about AR overall and embrace its unknowns — as well as any project unknowns we might encounter?”
Cue those tips and tricks I mentioned earlier.
But this is AR…
I don’t think dealing with AR is drastically different from other projects experience designers might encounter. The XD role at Slalom Build, and especially in consulting, is already one of constant discovery and evolution. We consistently switch between different screens, softwares, technologies, research techniques, and more. We’re adept at changing at will per project, per tech cycle, per need. It’s safe to say we are always adapting to new challenges, becoming experts as we go. With AR, the main thing to know is the challenge presents itself as both a physical space and hidden interface — giving us our first set of requirements and an ever expandable canvas at the same time. But that’s the fun part, right? RIGHT??? Before we can dig into the specifics, which can vary per project and use case level, there are a few things we should keep in mind to get up to speed and get our brains AR ready.
Reset Expectations
It’s important to note the most visible instances of AR and how most people outside the tech industry are introduced to this technology. Mass consumption of AR, at this point in time, is usually with a gaming focus–Pokemon Go and Oculus rift experiences are staged as splashy and visual pleasing entertainment with little environmental restriction. These examples are great for the sky’s the limit future-focused mindset we like to nudge clients towards. But for the value driven, user-focused results we strive to deliver within a manageable timeline, to say “hey, let’s build you a PokemonGo but with BLANK instead!” Well that’s the wrong place to start. To deliver value to a client and your company, you need to set proper expectations from the beginning and continually bring focus back to what’s useful and what’s just for show.
To properly set expectations, I’ve found it beneficial to start with “Why is AR useful here and how can it provide value to this experience?”
Well, luckily for you, through intensive research and much brain exhaustion, I’ve summed it up in 3 ways. AR can offer…
- additional content or context about a specific reality.
- interface(s) that blend seamlessly into a reality, providing a truly lifelike experience.
- shared learnings and interactions that are not constrained to a single person’s device.
Furthermore — AR experiences are easily grouped into 3 types:
Inform and Immerse
Using the environment to uncover location-specific information while keeping the user firmly planted in the space.
• Takes on a more educational or business-minded experience, can can be less interactive.
Explore and Entertain
Using the environment as a jumping off point for a pleasurable and experimental experience that enhances the surroundings.
• Takes on a more entertaining, gamified experience. Highly interactive.
Interaction into Action
Using the environment as the stage for a multi-step or continual process that will evolve and update in real time.
• Takes on a more action/reaction-oriented experience. Features various levels of interactivity.
Even outside of AR, this is a challenge designers face consistently: the desire to deliver bold, forward thinking visuals or bring that WOW factor in our interactions. Asking “Why is this technology or service useful and how can it provide value to this experience?” will yield the same positive results, which in the end might not be the sexiest or flashiest work.
Finding Value
More important on a client level is always keeping in mind why we are choosing AR over another type of tech. It’s often a matter of asking the right question. It’s not “why are we creating an AR experience?” but instead “why are we augmenting reality at all?”
Are we…
- shortening the length of communications?
- providing easily accessible content to increase productivity?
- bringing real-time data and context to points in space that can assist with making decisions on the spot?
- merely highlighting and elevating an environment to bring more pleasure to an experience?
Regardless of the answer, it’s incumbent upon us as experience designers to make sure any decision or additional feature stems from the overall goal of the AR experience. Just like a simple web project, we need to stay resilient and refute additions that aren’t going to add value or might distract from the overall experience, regardless of how sweet they are.
Business as Usual
Okay, but what about the actual work?
The nitty gritty design detail, based on my experience designing for a project that didn’t actually need design, here are a few things to keep in mind for your next AR project
Research: Focus on interaction within the environment and remember a physical environment isn’t necessarily stable.
Ask yourself “what outside factors will help or hinder the experience?”
Be tactile: Guerrilla prototyping with paper screens, making animations as you go (quick and dirty!) Don’t get caught up in jumping straight into After Effects.
Ask yourself “how can I experiment with visuals on and the off the computer, or outside of time consuming animations?”
Be prepared: For unique scenarios. A real-world environment is not as a stable as a desktop monitor so curveballs and compromise will come fast.
Ask yourself “what’s my back up plan?”
Be clear: Provide a glossary of new terms for the client so they know what you really mean when you say anchor or hologram.
Ask yourself “Do I know enough about what constitutes a hologram to help my client learn?”
Collaborate more: Chances are, your developers are new to this tech too, so why not tackle it together — no need for them to learn React if you can do the same thing guerrilla style.
Ask them “what’s out next-best option for proving this can work”
Use all the space: Keep in mind there are two planes to work with, the 2d device screen and the 3d projected reality. Make sure to leverage both to accommodate content.
Ask yourself “would I really want to scroll through all this…in the air?”
And Some Things That Might be New…
At Slalom Build, we follow a pretty strict project timeline — not necessarily time-wise but definitely in delivery. Before we enter any delivery sprints we get the lay of landscape, prepping deliverables like journey maps, user groups, stakeholder research, design systems — foundational stuff.
Since AR with spatial anchors is heavily location based, I was able to pinpoint a few new deliverables that assist in tackling new planes and general team understanding.
Environment Map: Where does the AR experience start and end? Where is a new screen or content piece triggered. Create an environment map to plan how and where AR pieces come into play.
Anchor/Feature Catalog: What’s an anchor? What’s content? What’s actionable? By defining and cataloging all the pieces in play with a map of where to find them, you’ll have a comprehensive UX/AR playbook.
Augmentation Logic: On the technical side, what happens when something is triggered? Or when a different device is in play? An AR experience can squeeze complex interactions into a few quick prompts or call outs, so documenting the logic to these interactions — and capturing an error case scenarios — will keep the confusion to a minimum.
Now, Go Forth!
As with all projects and products where user experience is involved, the devil is in the details, but there’s value to be added when we can tackle the unknowns head on. We might not always end up with a slick new interface, but being able to jump in and untangle a bit of confusion keeps the product team moving forward. My AR experience wasn’t the most comprehensive but the experience of not knowing where I would wind up in the end is something we can all relate to. And, ultimately, it made me more aware of the uncertainties of this emerging technology.
