Context-First Design for Augmented Reality

This article is a continuation of the ongoing series ‘A Quick Guide to Designing for AR on Mobile.’ Catch up on Part 1, Part 2, Part 3 and Part 4

What is Context in Design?

“A central tenet of Contextual Design is that any technology, product or system must be designed to support and extend its users’ work practice . If it does so well, it will be accepted and valued; if it fails to do so, it will cause dissatisfaction, frustration, avoidance and workarounds” — Encyclopedia of Human Interaction.

By definition, the noun context refers to the circumstances that form the setting for an event or an idea.

“con” — to join, “teks” — to weave/plait

The origins of this word come from the Latin word ‘contexere’ which means to join together by weaving.

We use context often in design. It’s invisible and immediately communicates intention without needing an explanation.

For example, the raindrops in the following apps are enough to communicate that it is raining without having to necessarily read everything.

Yahoo Weather, Apple Weather and Pokemon GO

Contextualism in Architecture

Contextualism in architecture is a principle of design in which specific urban and natural environments inform the design of a structure.

The two main determinants to the context of a building are:

The Physical/Natural Factors:

This refers to actual physical features such as the bend in a river, the height of a mountain or the shape of a tree

The Socio-Cultural Factors:

This could refer to the the site’s previous use or the current community and its people.

An example of both factors is at the Meteora Monasteries in Thessaly, Greece. The monastery is built into the landscape, as it is advantageous to natural defense and protects the inhabitants during times of unrest.

By analyzing, adapting and adopting factors of the environment, an architect can validate a building’s purpose.

The Challenge of Augmented Reality

Now what if an architect was given the task of designing for an unknown environment that could be anything- from a desert, a cliffside or a jungle? How would they be expected to create the best space possible without any understanding of the actual space?

That is the predicament of spatial computing. Spatial computing is a term that encompasses Augmented Reality, Virtual Reality and other mediums that involve being immersed digitally in space.

Designing for spatial computing comes with the reality that we have no control over where a person using the device will be. This is challenging because if we want our experiences to be engaging, our content has to adapt to a new environment each time and therefore we can no longer design with just a single environment in mind.

Current AR experiences on mobile often requires the user to define their space before they can get started. This adds a layer of friction since the content does not automatically adapt to the users surroundings, instead, it tasks the user with accommodating for the content.

An example of the houzz app where the user has to first define space and then manually place content

The good news is that we already design to adapt to our environment in some capacity. Take, for example, responsive web design. We define queries and conditions for a website so that it can anticipate any screen size.

Spatial computing requires this thoughtfulness, but for the real world.

Weaving The Real World with The Digital World

Regardless of the ambiguity we face in spatial computing, we will always have one vital piece of information, and that is knowing who our users are and what their needs and expectations are.

If we want to design meaningful experiences, we must understand and define what a computer sees and how it interprets that data back to our users.

This means authoring context that is responsive and smart enough to understand the world around us.

For example, if our experience involves generating a prompt on a chair, there are many factors to consider. The first being our innate ability to find affordances around us in the world that manifest into what we consider a chair.

A ‘stump’ is as much a chair as a ‘throne’, they both enable the possibility of seating.

We also have to overcome computer bias which is inherent in the technology present that captures and interprets data.

Examples of computer vision algorithms trained primarily on western constructs of a chair. If the data pool is optimized for a western standard, then the technology will always favor that.

Bias is the algorithm’s tendency to consistently learn the wrong thing by not taking into account all the information in the data.

Consider a variety of environments when authoring context as to keep experiences truly inclusive

How to Best Define Context

In order to optimize for what is ideal for our user, we have to clearly define the context to the best of our ability. We can do so by defining the following factors with as much detail as we see fit.

Physical Factors

Define qualifiers for what makes a chair different from other objects in a room

Socio-Cultural Factors

Be mindful of the human interpretation of things around us

The more thoughtful the context, the better we can make them work in any environment regardless of where in the world that may be. This in turn, really extends the value of our designs. This is also true immersion because it adapts contextually and responsively to the space around the user.

By being contextual and responsive, things feel more intuitive to the user.

Tools for Authoring Context in Spatial Computing

At Unity we are solving this exact problem with Project MARS. Project MARS is an authoring tool that lets users design spatial computing experiences using context.

In MARS the author defines the context for their content and then tests how their experience might work in a variety of different scenarios.

The author can create a context that will then read data from the real world through an AR enabled device (such as a phone or a headset) and interpret it into something desirable.

The author can also simulate in real-time inside Unity, how their users might interact as they discover new surfaces

Project MARS also makes, what would otherwise take significant time and effort, a lot simpler by transforming it into easy to use visual tools. This creates less mental overhead when embracing a new type of context-first authoring.

Ranges help define a variety of potential candidates instead of sticking to a single set of sizes.

Project MARS is currently in closed alpha, but we are looking for dedicated teams to partner with who are trying to push the bounds of spatial applications. Learn more about Project MARS here.

Technology moves fast and with more data available to us now than ever before, contextual design will help make it meaningful. After all, it is our responsibility as creatives to help lead new technology forward through thoughtful design.

Interested in reading more about how things work, read Designers Guide to Hardware and Software for AR.

Special thanks to Sidra Mahmood, Ash Huang, Devon Ko and Brendan Ford for their support and review.