Eye-tracking in Usability Testing

There are a couple of common design patterns that can be seen across the product portfolio here at CA Technologies. These patterns include monitoring dashboards, configuration data grids or topology maps.

Recently, I wanted to evaluate, how we can use eye-tracking technology to test and improve usability of these common patterns. I have previous experience with the technology, but the enterprise environment represents a different challenge when it comes to user research. My goal was to start small, borrow the camera for one-time only event, and have a hands-on experience with the technology.

Device

Thanks to Circus design, we were able to use the Eye tribe device and test the technology on an internal event.

Eye Tribe tracker is a small and handy device.

Testing

I had a couple of initial ideas about usability testing methods that I wanted to evaluate.

1. First Impression study / Time-out test. After looking at the page for 20 seconds, what information was the user able to digest? What captured her attention? This is an important test for every dashboard or monitoring console.
2. Findability test. Can users quickly and easily find the information they need? Was the page visual hierarchy helpful in this task? Can they spot all the the alerts or warnings?
3. Iconography and Semiotics. Can users understand the symbols, and how long it take them to interpret the symbols?
4. Quantitative studies. Looking at the aggregated data from multiple users, what are the areas of most and least interest? Can we suggest visual hierarchy improvements?

Unfortunately because of technical difficulties, we were not able to prepare detailed task scenarios in advance. We simply played with the camera with my colleagues in a noisy environment and settled down with the basic time-out tests.

Performing basic time-out usability test.

Testing the device on Prague Engineering days internal event.

Results

Data grid alignment confusion. The user was distracted by the wrong form label position and unexpected value of the "Version" field.

Sign Up for the trial test. Although button is visually clearly separated, the page layout causes a little confusion with another link called "Sign in".

Conclusions

Eye-tracking camera has ben proven useful technology and my team was able to recognise the benefits of the usability testing outputs. The issue with camera drivers prevented us preparing more sophisticated test scenarios in advance, but this problem could be solved with a better preparation.

We found the camera calibration to be the limiting factor for real-life use. The software was often losing the eye-gaze tracking after just subtle body movement.

Despite few drawbacks, I would recommend the device and we are now considering using the camera on real-life future usability testing.