Neuroergonomics - what is it and why now?
From concepts to application
Investigations about the interaction between a worker and his workplace are almost as old as work itself. These interactions are investigated for the purpose of efficiency in production, logistics, cutting down costs, etc. Human posture became very relevant for “lifetime employment” companies, like car manufacturing and others. If a person developed a health condition due to bad posture, this results in bad PR and even more importantly, the company would become liable for compensation. This constitutes an old branch dealing with posture and movement: ergonomics. Though its roots stem from the 5th century BC, it has flourished with 19th-century industrialization and it is today an inevitable part of the design of not only workplaces but also car seats, chairs, machines. It took about 100 years for its first “major upgrade”, Neuroergonomics, to appear.
What is exactly Neuroergonomics?
However, unlike the largely “physical” component or ergonomics, the “psychological” part of the equation remains largely under the radar, and only recently started to get attention. What is this psychological part? Well, humans are not robots and their beautiful complexity of mind can constitute a problem for production or other types of work. Issues such as motivation, emotions, mental workload, and in general, cognitive effects of worker’s interaction with work needed to get the attention. This came to be a separate and very recent branch of science, christened “Neuroergonomics” by its creator, Professor Raja Parasuraman. He defined it as the study of the human brain in relation to performance at work and everyday settings (Parasuraman, 2003; Parasuraman and Rizzo, 2008).
What was the novelty of this approach?
The important breakthrough that may not be obvious at first was, that giving a status of a new scientific branch meant that it was treated in its entirety.
What does that mean?
For one, the merely psychological interpretations of a workplace now got investigated in more depth and most importantly, they started to get quantified. Initially, methods were dominated by questionaries, which are mostly of subjective nature. They were inherited from classical ergonomics but dealt with the perception of various parts of the work process. However, neuroscience offered a completely new array of methodological options, allowing for the first-time objective parametrization of emotional and cognitive states, paving the way for the quantum leap to come.
Making a transition to new methods in Neuroergonomics
One begs a question: why didn’t we start with well-grounded neuroscientific (quantifiable) methods in the first place? How did an obvious choice like this evade everyone’s attention until now?
Well, it didn’t. Similar ideas can be traced back to the sixties, and studies trying to achieve similar goals set by neuroergonomics (though still not under that name), e.g. Gevins, 1995. But treating these complex issues required gathering physiological data (the “quantify” part) in a relevant environment WITHOUT distracting the subject under observation. That is what the bottleneck was — the equipment for this was missing. When the rapid technological advancements from the 2000s removed that obstacle, there was a paradigm shift and we are witnessing the flourishment of a new exciting field of quantified neuroergonomics. Let’s pause here for a while, to provide a few basic comments what is the nature of physiological data we wish to observe, their relevance, and the equipment and methods used. We will do so in the “Physiological signal box” below, which also explains the recent technological trends and how they have put the spotlight on a few physiological signals of interest. These tools provide us with the option to make the advocated transition to quantifiable Neuroergonomics of the 21st century.
Physiological signal “box”
Here is a short overview of data types, how they are relevant for neuroergonomics, previous obstacles and recent trends.
EEG: measures brain activity of neocortex (most developed part of the brain, responsible for who we are as humans).
Relevance: direct tracking of mental states, emotions, mental workload, focus, etc. Great temporal resolution — relevant for real-time applications.
Bottlenecks: for a long time too bulky, expensive and stationary for these applications
Recent trends: EEG devices becoming smaller, cheaper and completely mobile (e.g. the author co-designed one such system). There are readily available solutions on the market.
fMRI: measures brain activity by detecting changes in cerebral blood flow
Relevance: neuronal activations are coupled with the blood flow — therefore the method offers insights in activations of brain areas (with respect to some defined condition)
Bottlenecks: too large to employ in relevant conditions; subject needs to be placed in a fixed location; in many cases available only for medical applications (high price); low temporal resolution
Recent trends: more available in general research; used to reveal population trends and reveal fundamental brain functioning schemes.
fNIRS: functional Near-Infrared Spectroscopy — used for functional neuroimaging.
Relevance: another (optical, complementary to others) method to investigate brain activity.
Bottlenecks: previously too large for neuroergonomics applications
Recent trends: becoming more available, even on consumer level; increased comfort — better design;
GSR: measures changes of skin conductance
Relevance: skin conductance changes very fast as a result of emotional arousal
Bottlenecks: expensive, not available outside of e.g. police applications
Recent trends: many appearing solutions, mobile, inexpensive. Available even for mobile phones (via audio jack).
Eye tracking: tracks parameters of the eye movement, tracks eye gaze.
Relevance: where a person directs attention over time
Bottlenecks: not mobile, expensive
Recent trends: several mobile solutions already available, prices also going down.
Heart rate (and heart rate variability): tracking each heartbeat
Relevance: enables tracking stress and other conditions related to mental and physical workload
Bottlenecks: not so comfortable to wear
Recent trends: available on the market as consumer wearables such as wristbands.
Video monitoring: camera systems
Relevance: behavioural measures of internal human state — posture, movement, etc.
Bottlenecks: software for quantifying movement missing, camera fixed
Recent trends: cheap systems comprising both software and hardware available (e.g. Microsoft Kinekt)
Also: tracking fingers, respiration, facial expressions, head movement…
With new methods at hand, what is it that we wish to achieve?
Sometimes, it is good to zoom out and come back to basic questions like “why are we doing these things in the first place?”; otherwise, measurement equipment and quantification methods could easily become their own purpose, which is obviously wrong. So, coming back to the original goal: the reason to include physiological data into the process of making and quantifying the workplace is to suit basic human needs while optimizing the task being performed.
To this end we need to combine physiological data, choosing the relevant subset of sensors to achieve this goal that is also applicable to the concrete task. For instance, we cannot hope to have a camera system always recording the face of the study participant from the same angle if he is moving.
Prioritizing equipment comes from the most basic needs of a study. It can be, for instance, tracking attention level and its oscillation during time or task. Having confirmed the ability to track the feature of interest (in this example attention), we can then see what a minimal and most applicable subset of sensors provides us with this ability, making a study easier and participants more comfortable. And why not, future industry applications closer to reality.
Some successful neuroergonomics experiments involving brain activity observation
While still in its infancy, we have a number of studies existing today that employ quantification as a trademark of neuroergonomics. I will name just a few of my personal favorites (but feel free to contact me with your applications!)
Wascher et al. discovered age-related differences in motivation when confronted with monotonous workplaces, based on EEG recordings.
Krebs et al. dealt with the issues of learning a complex task (robotic hand control) and used the PET scan to associate and detect differences in the brain.
Karni et al. found evidence for multiple stages of learning using fMRI, characterized by different time scales, gains, and patterns of activity in the primary motor cortex. An important feature of this study was that experience-related brain changes were measured over an impressively long period of training, 3 weeks.
Although PET and fMRI are not applicable during the work process itself, in previous examples they still proved valuable for determining the objective brain state changes.
Lewis L. Chuang’s group recently investigated the demands that steering places on mental resources in a very insightful EEG(ERP) experiment.
An excellent study by Lamble et al. investigated driver’s attention when using mobile phones. Even though tasks were supposed to use different cognitive resources of the brain, e.g. visual attention employed for driving and auditory for speaking, these modalities proved to have “cross-correlation”. Another great text from my colleague Bogdan on mBT blog elaborates on this issue in more depth.
Mijovic et al. replicated a whole factory production workplace, investigating EEG, reaction times, body posture, and showing how attention oscillates during a monotonous task, a first step to addressing this issue further. This represents a step forward in mimicking work conditions faithfully.
Examples are numerous, but I have to limit myself. Gramann et al. recently published a book describing comprehensively recent developments and future trends in neuroergonomics. We can also witness increasing demand for such research: there is now a dedicated Neuroergonomics conference; the last was held in Paris, and the next edition is to be hosted by Drexler University.
What does the future hold?
We are only seeing the surface of what wearable technology offers us for neuroergonomics challenges. As in other domains, technology will continue to evolve, from research paving its way further into application space very quickly. When there is a need, there is a way, the old saying tells. Like reaction times (RTs) for train operators quickly made their way to application, Event-Related Potentials (ERPs) may be the RTs of the 21st century (SJ Luck, pg.22) and so will be the case with other physiological data thanks to the avalanche of wearable devices that are becoming available.
Here is where the story becomes very interesting. We still have no gold standards for mental states, including attention, emotions,... What we do have is statistically verified (we know when someone is sad or happy), but are our current methods indeed the best? Or there is a better and more objective scale for all of this? You are guessing my opinion by now.
Are there multiple types of mental workload, or is someone “just” mentally tired? Again, you are probably guessing that I am closer to the former.
We started from a workplace-worker interaction, but the list of tools we are discussing is suggesting a much broader application scope. From personal well-being to emotional interface to the world if you like, we will see in not so distant future. One step at a time, because the road ahead of us is very long: we first need to adapt experiments we consider reliable to simulated working conditions, then, translate them to real-life workplaces, and finally, assess their usability. Will someone use the wearable that reveals his (in)effectiveness? If you asked me 10 years ago, I would shout “NO!”. But how much of our private data have we voluntarily given up on for the benefits of “data age”? So now I think for myself “Yes, I would love to know what affects my effectiveness, show me the product that can measure it”.
But what about the workplace evolving itself, driven by AI and automatization? Whatever we do, we must keep the vision of the future aligned, otherwise, we may create methods for problems that don’t exist anymore… Enough philosophy for today, back to work now.
