Augmented Reality — The Human Dimension

Co-Authors: Kay Stanney

With its potential to substantially enhance productivity, augmented reality (AR) is poised to become the point-of-need training and operational support tool of choice for manufacturing, service-based, and military organizations across the globe (Romeo, 2018).

AR technologies can facilitate human performance through presentation of virtual overlays superimposed on real-world objects and environments, which allows for guided hands-on embodiment of psychomotor skills and immersion in contextually-relevant scenario-based learning and/or real-time coaching experiences. Questions remain, however, with regard to the effects that AR may have on a worker’s ability to process and retain information.

Early research outcomes are demonstrating that humans perceive and process information differently in AR than in real-world environments (Wang, Huang, Liao, & Piao, 2018). Within AR, there may be distractions and perceptual deficits caused by limitations in display graphics and optical capabilities, restrictive field-of-view, and other constraints that may divert attention from presented material.

Due to these limitations, viewers may erroneously code AR cues or develop inappropriate accommodation methods while viewing content in AR headsets. Mismatches between capabilities of the human visual system versus AR display can also be a recipe for headaches, eye strain, and a whole host of other physical symptoms that may affect viewers both during and after exposure.

Bottom-line, limitations in AR headsets can negatively impact (see exclamation points in the figure) every dimension of human information processing and ultimately the decision execution that AR is ultimately designed to enhance.

A host of potential concerns associated with AR display design, optics, and graphic capabilities should thus be considered to ensure targeted performance gains are achieved. Some issues that must be considered, include:

• Depth planes in AR head worn displays (HWDs) are fixed whereas the human visual system is adaptive. This kind of limitation in support of stereo-acuity can cause a distortion between the real and virtual worlds, which may result in misrepresentation of space, difficulties in localization and understanding relative depth, and even optical illusions for the viewer.
• Low resolution and limited contrast can negatively impact object recognition, as well as the ability to track objects in AR and lead to failure to perceive or misinterpretation of cues critical to human performance.
• In traditional heads-up displays, it has been observed that there can be issues with occlusion, attentional tunneling, and inappropriate reliance on artificial cues (Foyle, Dowell, & Hooey, 2000). There may be similar issues in AR HWDs related to perception, allocation of attention between real-world items and AR cues, and other such human information processing issues that could diminish decision execution during operational support or learning, and training transfer to real world tasks during AR training scenarios.

Taken together, these concerns have important impacts on workers’ ability to process information in AR HWDs, impacting attention allocation, sensation and perception, and ultimately impacting high-order cognitive constructs such as situation awareness, decision making, and decision execution.

If industry leaders are to realize the full scope of productivity enhancing possibilities with AR applications, it is important to fully understand the impact of AR on human information processing so we can reduce negative effects, and boost training and performance gains.

Preliminary findings from AR human participant testing in our labs provides some initial guidelines for maximizing the operational support and training potential of AR applications:

1. Spatialize Audio — Spatializing audio is a key component of any AR application, as it can direct attention and support situation awareness. This feature allows audio information sources to be placed at multiple depths and locations, creating a more intuitive design that can guide perception in AR-supported environments. However, it’s important to ensure any audio design is appropriately placed, as misplaced audio cues could actually add to disorientation efforts rather than support perception.
2. Minimize Visual Distractors — Two-dimensional windows, icons, and menus should be a thing of the past. AR is so much more than just 2D information on a screen. It’s the use of a virtual world to augment our own physical real-world; to realize the full potential of AR, expand your thinking beyond the windows-icons-menus 2D user interface paradigm.
3. Choose your AR device based on tasks — If your tasks require hands-on work within 2m of the worker’s body, consider an AR HWD device with multiple depth planes. HWD with multiple depth planes allow virtual information to be presented at multiple depths, more closely mimicking the human visual system, and therefore may have less negative impact on human information processing, as well as less potential for headaches and other adverse physiological effects.
4. Align operational environment to AR HWD device — Not all AR HWD devices are created equal when it comes to environmental factors. Ensure you understand critical characteristics of your operational environment when choosing a deployment device. Consider factors such as ambient temperature, lighting, noise, and movement within the operational space (both the worker’s movements, and that of any other persons or objects in the space).

Augmented reality holds great promise to bring training and operational support to the next level. Once we fully understand the impacts to human information processing this unique technology has, we can design our applications at the highest level and reach for the stars!