Gaze Tracking Like a Gorilla

One reason VR analytics is so much richer than standard video or web analytics is that content creators can know where each user is looking during the experience. On other platforms, this information can only be gathered by using special equipment to observe a user’s face while they’re interacting with the application, and this only tells you where a sample of the population is looking. With VR, you can know where each individual user is looking during an experience, to learn what’s interesting and what’s ignored, what worked and what didn’t.

Except you don’t actually know where the users are looking, just where their heads are pointing. Actual tracking of eye direction hasn’t come to consumer VR headsets yet, so the current state of the art is to measure the orientation of the headset and project a line out from between the user’s eyes. Whatever that line intersects in the scene is interpreted as the object of the user’s gaze.

We know from personal experience that head orientation is not a good indicator of where a person is looking. There’s even a move in football called the “head fake,” where a player tricks their opponent into thinking they’re moving in the direction their head is pointing, while actually moving in the direction of their eyes.

And knowing where a person is looking is important for human interactions. According to the cooperative eye hypothesis, the sclera, or white part of the human eye, exists in part to provide contrast with the iris so that we can easily see where the eyes are looking. Experiments have shown that human infants have no trouble following a human’s gaze, whereas primates, who have darker colored sclera, only follow where the head is pointing.

It’s still useful to know where every user’s head is pointing when measuring the success of a VR experience or movie. One of the challenges of 360 video is that filmmakers no longer control “the frame” and must rely on other techniques to direct the user’s attention to important parts of the story. Head tracking helps them know if they were successful or not, and creators of interactive apps can use this data to inform key design decisions. But when eye tracking becomes available in consumer headsets, VR analytics tools will finally be able to track where users are looking the way humans do. Until then we’re still gaze tracking like gorillas.