How using virtual reality with integrated eye tracking can create training that improves speed, accuracy and safety

It is said that beauty is in the eye of the beholder. In the case of eye tracking, a type of research used to record and analyze where people focus their attention, one could argue that, actually, improved skill and safety are in the eye of the beholder.

Mike Bartels, senior research director at Tobii Pro Insight, a research company, said eye tracking is used in a variety of contexts including academic and commercial research and can help improve training practices.

“This technology is valuable for any type of application where you’re interested in understanding attention — which applies all up the chain of professional performance and skills transfer,” Bartels said. “We’ve done research on how surgeons perform operations, how foundry employees pour melted aluminum, how a professional race car driver navigates the track — it’s a broad field even in professional performance with quite a few different applications.”

Most recently, researchers from Tobii Pro Insight conducted an eye tracking study at H&H Castings, a metal foundry that supplies aluminum castings, that may help accelerate onboarding of new workers and reduce the risk of accidents.

For the study, a sample of six foundry workers in the melt department at H&H wore special glasses that tracked and recorded their eye movements as they completed their regular tasks. The recordings were then analyzed by software to create a database of eye tracking videos that can be used to train new employees before starting them on the floor.

Safety First

The melt department deals with molten aluminum that can seriously injure or kill workers, which makes training new hires difficult. H&H System Manager Jacob Hammill said there are many risks involved during training, such as molten metal pouring out of ladles and metal swinging around on big tracks. He said the biggest benefit from eye tracking technology is it allows H&H to conduct training without endangering new hires.

“Now we have this database where we can show them right off the bat what they’re going to be doing and what hazards are around them without wasting any time on the floor,” Hammill said. “It’s helping keep production going without having to stop and reiterate things to a new employee.”

Tobii Pro’s Bartels said he sees this issue in a lot of industries. He said it’s hard for operators to fully explain what they’re doing when training someone for their position. But with the eye tracking videos, new hires are able to listen carefully while seeing tasks performed through the eyes of the expert employee.

“Somebody can come in on their first day and watch the performance of the task at hand and be able to see everything that a person should be looking at and paying attention to in order to perform the job safely and efficiently,” Bartels said.

Hammill added that employees enjoy being able to get insight into their errors by playing back the videos. “It’s cool because up until that moment they never had a chance to go back and look at what they had been doing that day through their own eyes,” he said.

Bartels and Hammill estimate the melt department will save 400 hours of training per year, an average of two days of training per employee.

Digging into Data

The recorded videos also allow users to identify behaviors associated with optimal performance. At H&H, analysts noticed that successfully completing the task of pouring the molten aluminum was associated with a specific attention pattern. Successful workers were visually focusing on the target into which they were pouring the aluminum, keeping their head and body perfectly still, with little eye movement aside from focusing on the target. Analysts saw spillage when employees didn’t fixate on the target and keep their heads still, Bartels said.

Hammill said in the past they would associate a spill with what they believed was a body error, such as too much forward motion with the ladle. But when they got the data back, it was evident it was because of eye movement. “You could see his eyes gaze toward the wrong part of the mold and — boom — the mispour happens,” he said.

Bartels added that the extent of visual focus required for some of these tasks can cause attentional strain resulting from employees focusing too long, which can cause them to start making errors. With the data, they can recommend the optimal shift before a break is required.

“We’re really able to get into the weeds of this attention data, identify the behaviors that are associated with the best performance and communicate those to the staff of the company for new training materials,” Bartels said.

Although H&H is still in the early stages of implementing this technology into its training program, Bartels said some clients have been using eye tracking technology for two or three years.

“In one form or another, eye tracking has been around for 100 years, but it’s only really been in the last five to six years that the technology has been small and unobtrusive enough to be used in a realistic work setting,” Bartels said. He pointed to a car manufacturer that has been using the technology for a few years and has reported a 50 percent decrease in visual inspection errors since its implementation.

He added that eye tracking technology also is integrated into virtual reality headsets to track and record where people are looking as they are performing tasks within the virtual environment.

Ave Rio is a Chief Learning Officer associate editor. Comment below or email

Tags: data, Eye Tracking, H&H Castings, research, safety, technology, Tobii Pro Insight, training, virtual reality

Originally published at on December 12, 2017.

Additional note
Did you know that Snobal’s AI assisted VR software captures ALL user interactions in the virtual environment including eye tracking? Capturing this data unlocks hidden value in virtual reality when used for design, design testing, stakeholder engagement and of course training.