Thinking about the User Experience in AR Experiences

9 min readFeb 19, 2019

Tips from a UX expert on how to plan, build, and perfect an immersive AR experience. How to keep users engaged, prototype, and make something awesome.

AR is coming out of the hype-cycle and has begun to see innovative real-world uses such as the Ikea Place app. I thought the time was ripe to share some trade secrets about building immersive AR experiences.

I’ve been designing experiences for more than a decade. After starting in design, branding, and content, I moved to product design and development. Now, I explore the intersection of digital and physical through immersive AR.

Together with my team at Jam3, we’ve built experiences for everyone from Google to Facebook, Universal to Warner Brothers, Ford to Hyundai.

In this article I’ll give you a roadmap for bringing your AR dream project to life, or better optimizing an existing one.

These are the key learnings and best practices I’ve picked up in my work — along with some stuff I wish somebody had told me when I started.

Don’t lose the plot: Start with the story

A captivating narrative is the core building block of any engaging AR experience.

Think of it as a blueprint to a finished house. The story provides the framework that decides the depth and scope of your project, as well as the components needed to bring it to life.

It’s tempting to jump straight into working with an exciting piece of tech before thinking through the reason for doing it.

Using technology as the starting point of a project can lead to delays, confusion, and even an end product that doesn’t quite fit its purpose.

To continue the architectural metaphor, it’s a little like deciding to build a house because you bought a new extension ladder.

It’s all fun and games — until you realize you forgot the windows.

The story will strongly inform the palette you’ll work with. This can range from wide-scope considerations, such as whether AR is the right tool in the first place, down to the fine details like character, UI and sound design.

Something worth noting is that narrative structures work a little differently in an AR experience than they would in your usual novel or TV-series.

The “opening wow”

AR has the best results when paired with a slightly tweaked plot structure, or by using a device we call the ‘opening wow.’

A standard plot structure, graphing the rising and falling of tension within a story, looks something like this:

Movies have trailers to hook a user’s attention and interest.

AR games and experiences rarely have that marketing budget. So they have to wow — a thrill or emotional hook — a user early in the experience.

For example, one Game Developer ,seeing low experience play-through, moved the most visually spectacular part of their climax to the start of their experience.

The result? They saw play-through increase dramatically.

That ‘wow’ is most effective if served up early — even as early as within the first 15 seconds.

Here’s how that might look:

Nonlinear experience

Another key difference is that a story in an AR experience isn’t as linear as it might be in a novel or movie.

Of course, as a storyteller you still guide the user through a narrative.

But the cool thing is that the medium allows them a level of freedom to interact with what interests them in the space you’ve created — whether that’s a character, object or hotspot within the world.

To take it a step further, AR makes that interaction physical. That space you’ve created exists within the real world.

Your real-life movements decide how you experience the game. For example, this world might exist within the boundaries of a table-top. To zoom in you actually have to get up close to it.

The effect gives a kind of dimensional immersion. Every user can have a personalized experience, and every play-through is unique to that user’s style and pace.

Your attention, please: Tips for keeping the user engaged

The storytelling power of AR carries with it an equally powerful responsibility — keeping the user engaged and rewarded.

This challenge should be considered in the early stages of building. It should also be revisited consistently in their shoes, tweaking and remodeling accordingly.

It can be all too easy to focus on innovation so much that the actual fun and utility of the thing gets left by the wayside. While research is relatively limited, there are some great reports highlighting the constraints and opportunities of AR.

1. Building for laziness

My team and I use an approach we like to call “building for laziness.”

This approach assumes that if the experience gets too abstract or hard to figure out, the user will switch off.

There always needs to be a clear goal and method of progress towards it, or else fun can quickly become a frustration.

One of the most common methods to build for laziness is using elements of the game environment to guide attention and prompt action.

There are many ways of approaching this based on the type of experience you’re looking to build.

These cues can be as subtle as faintly highlighting an object of interest with an action prompt that appears when you’re in proximity of it, or having a character beckon you from a distance. The idea is to direct the user’s attention using features within the parameters of the environment.

Of course, you can achieve a similar result using the UI. For example, you can build a taskbar that lists active objectives or a ‘help’ screen, but this method can look and feel clunky and unrefined.

Google discovered this in its early explorations developing Daydream.

2. A multi-sensory approach

The cues don’t need to be visual either.

Often a multi-sensory approach works best. For instance, you can add a hum that increases in volume the closer you get to a highlighted object, or a section of narration that hints at what to do next when interacting with an object or character.

3. Gamifying the experience

Another pathway to retaining engagement is by gamifying the experience — because who doesn’t love being rewarded?

The addition of points, collectibles, and tangible progress might seem small, but it can add a whole layer of depth.

Bonus points if you’re able to add cooperation and competition, there’s not a lot better than sharing an experience (or an intense rivalry) with friends.

And that’s not just sharing your score with friends and challenging them to beat it. With AR you’re able to digitally and physically sync with others as you progress through a storyline and complete objectives together.

My team and I have learned a lot of things from user feedback.

One is that not being able to enter and exit the experience freely, sucks. Life happens unexpectedly, so the freedom to jump in and out at any time is crucial. Also, for some people an hour of AR can be a bit much, so it can be a useful feature for them too.

The bricks and mortar: Building your AR experience

So, you’ve got a compelling story that’s perfect for AR — but how do you build it?

1. The right team

A good starting point is making sure you have the right team for the job. Software design is a team effort, so multi-competency is paramount.

Not only should the people on your team be specialists in their specific roles, but they should also be aware of the end product as being a whole that is connected to everybody’s individual efforts.

Team members will be relying on each other at every step of the way, so it’s important that they’re communicating consistently. It’s much easier to change the figurative light bulb if everybody understands how it’s done.

2. Planning the process

Next up, planning. Plan your process.

Run sprints often, and set aside time to review them.

It might seem glaringly obvious, but half the project is in the planning. Get it right and you’ll save yourself and your team a lot of stress and effort.

To add to that, ensure that more than enough time is allocated to prototyping and testing. This is a process that you’ll want to repeat. It’s rare when things go perfectly the first try, and it’s often not quite there the second and third tries either. Prototype, test, feedback, rinse and repeat.

3. Design-oriented mentality

My team and I like to have a design-oriented mentality from the get-go. This is just as much about the experience looking good and functioning well as it is about it being fit for purpose. There are three main areas of design, which overlap and inform each other:

An AR experience is generally equal parts narrative, game, and product, so it helps to think of them as intersecting disciplines which affect each other.

Keeping that in mind, we can break the rest of the build down into three rough sections.

Three parts of building an immersive AR experience

1. Blueprint

This is the conception stage, where the ideas are laid out on the table and built upon. Here is where you will decide on a concept and how to approach it.

This stage starts with applying beats and plot points to your story concept, as well as how and where they’re encountered. From there you can start to build a world around that story with concept art, and then apply game architecture (how it would play out functionally) on top.

2. Prototype

Now you have the blueprint ready, you can start to build.

The three main considerations here are recording 3D audio, running collaborative sprints (including design, UX and motion), and testing.

As above, this stage entails repetition — continue to prototype, test and refine the pieces until they’re at a level you’re happy with.

No one has figured out how to best test AR experiences, but some thoughts are starting to emerge.

3. Produce

Now put all of the pieces from the prototyping stage together to produce an Alpha version. The Alpha should contain all your core functionality, level blocking, and the lighting and texture rig.

Then you can take the Alpha, put it through a feedback cycle, refine it and build a Beta version. At this point, the experience should be 99 percent complete, with only minor adjustments to be made. This means full textures and lighting, animations and interactions should all be ready to launch.

Now you can get together your team, plan a roadmap for the launch including publicity and press kits, and maybe even have a party. Just don’t forget to gather user feedback for next time.

I hope this has been useful and that when designing the best immersive AR experience you can, you have as much fun as my team and I do.

Phill Dodd is Director of UX at Jam3 Toronto & LA.

Jam3 is a design and experience agency that partners with forward-thinking brands from around the world. To learn more, visit us at




Jam3 is a design and experience agency that partners with forward-thinking brands from around the world. To learn more, visit us at