Measuring User Engagement in VR
Compared to platforms like web and mobile, virtual reality offers an unprecedented level of data to measure how users are behaving. Heatmaps are a great way to interpret this data, but they must be viewed on a case-by-case basis and their qualitative nature doesn’t allow for easy hypothesis testing or comparison between different experiences.
One of our goals at Vrtigo is to create tools to help content creators and distributors quantify user engagement, using data unique to VR. In this post, we’ll discuss one approach to defining user engagement based on the spread in the distribution of where viewers are looking.
Introducing Audience Focus
Some VR experiences are created with the goal of immersing the user into a world they are meant to explore. The content creator doesn’t want everyone to look at the same thing. Other experiences are meant to direct viewers to specific areas at a certain times. Between these two extremes is a spectrum of viewer dispersion.
Audience focus is a unit-less metric that quantifies the degree to which viewers are all looking in the same direction. A value of 1 indicates that everyone’s headset is pointing the same direction; a value of 0 indicates that everyone is looking in different directions.
To demonstrate this idea, we’ve included an example of the audience focus measured on the RecoVR Mosul video from The Economist VR. This video takes viewers through a virtual museum containing reconstructions of artifacts in Mosul that were destroyed in 2015.
At the start of the clip, the viewer is transported to a room with multiple videos playing and museum pieces displayed. The heatmap indicates that viewers are exploring these areas, and the audience focus for this scene drops to around 0.45.
It then cuts to a room with only two interesting features: a marble tablet and an embedded 2D video. As there are fewer things to explore in this room, the audience focus begins to increase. Finally, it cuts to scene where the narrator calls attention (audio cue) to a sculpture and the audience focus nears 0.8 as the viewers converge on the object being discussed.
High vs. Low Audience Focus
To discover how filmmakers persuade viewers to look in a single direction, or to explore all 360 degrees, we analyzed several thousand videos to find patterns of high and low audience focus. The data for six representative videos are shown below.
Some takeaways are:
- When the focus is high, most viewers are looking straight ahead, where the action is. This is not surprising, based on what we know about where 360 video viewers tend to look.
- In rarer cases, high focus is the result of a moving object or person drawing everyone’s attention (as is the case with the ballet video shown in the top right of the figure, where one ballerina is continuously moving throughout the video).
- High focus can be maintained by using rapid cuts so that viewers don’t have a chance to explore the scene before it changes to a new one.
- Videos with lots of action (dancing, racing, boxing) typically have high focus.
- Videos with fewer scenes and no audio or visual cues tend to have lower focus.
- Videos with large groups of people or more than one main character encourage exploration and have a lower focus.
- Nature and space exploration videos usually have a lower focus.
A couple of themes emerge. Videos with lots of action usually have only one thing to look at and the scenes often change too quickly for viewers to explore them, so audience focus remains high. Videos with low audience focus tend to place the viewer in the middle of a scene with predictable movement and ample time to explore.
One can dig deeper, but this gives an idea of what’s possible when you begin to quantify engagement based on sensor data from VR headsets. In the coming months we’ll be releasing more VR viewer behavior metrics.
Vrtigo is the next generation VR analytics platform for content creators, editors, producers, and marketers.
