How to define metrics for augmented reality, mixed reality and virtual reality

Alternative reality (AltR), including augmented reality (AR), mixed reality (MR) and virtual reality (VR) has presented some new challenges when recording metrics. Product managers, user experience designers and developers all have issues knowing how and what to capture.

Digital product development relies on iteration, learning from users reactions to make improvements. Aside from direct observation and questioning, our best learning comes from collecting large amounts of data as users experience the product.

While some data points overlap with website and app metrics, measuring activity in alternative reality experiences requires attention to a few new areas. AltR experiences present different challenges, more akin to games.

Learning from years of website and app data collection, we can bucket our metrics into similar logical groups.

Understanding entry points

One dataset, much like websites and apps, helps understand how the user found their way to the experience. This includes all the standard acquisition metrics:

• Direct
• Search
• Social
• Campaigns
• Referrals

Understanding the audience

Also similar to traditional digital audience analytics we need to understand the user as much as possible:

• Demographics
• Interests
• Geographic

As well as understand their equipment:

• Hardware (computer, mobile, headset)
• Software (OS, AltR source)

Understanding the environment

Unlike virtual reality, augmented and mixed reality have the added layer of mapping the territory. Capturing elements in the environment can enhance the dataset. Some technology only captures surfaces and shapes while others include image recognition to interpret the nature of the objects themselves. Depending on the data available the following metrics can be useful:

• Objects mapped
• People mapped
• Locations mapped
• Dimensions of the space

Understanding behaviour

Whichever form of AltR environment, the interaction models vary a lot from traditional digital properties.

Events

AltR presents the users with a number of options or activities usually understood as an event tree. Each action a user takes presents a suite of possible reactions by the system.

• Passive events triggered (such as staying in a certain state)
• Active events triggered (such as selecting an item or walking into a room)
• Scenes loaded
• Characters loaded

Flow

AltR follows storylines controlled by the user. Capturing in what order events take place and the amount of time spent in particular states form a core part of analysing user behaviour.

• Sequence of events over time
• Heatmap of user focus
• Object focus over time (such as picking up, viewing from various angles)

Entrance/Exit

Analysts need to understand the state of the world when a user arrives and leaves. At present most experiences start in a single state, however, as they begin to connect and mobile augmented reality grows, users will enter in different states based on factors like location, demographics or interests.

• Entrance state
• Exit state

Interaction

In augmented reality, mixed reality and some virtual reality environments, users interact with physical objects. Capturing interaction with these objects really represents a linking of event data as outlined above with environment data, also above, created new compound metrics.

Current analytics landscape

Few native AltR analytics platforms exist at the moment. Retinad includes heat-maps and other AltR-specific data capture. But little else stands out in this nascent field.

A number of game analytics platforms, such as gameanalytics.com, also exist which cover unity and unreal engines among others. Teams can adapt these dashboards for AltR expereinces.

Alternatively, most web analytics platforms support the creation of events. These can be customised to relate to the necessary event states outlined here. Google Analytics covers all the basics and is extremely powerful for customisation as is Mixpanel.

Conclusion

Much as web analytics has grown and changed over time, AltR analytics will also evolve. For now, product managers, designers and engineers need to publish findings even as the inevitable new analytics software landscape emerges.