Why Spatial Audio is Important for XR

Amanda Galemmo
Nov 13, 2019 · 4 min read

Mixed reality has presented to us new ways to teach skills and solve problems previously impossible. Whether onboarding new employees or assisting first responders remotely, the use of VR and AR allows developers and creators to assess situations in innovative ways. In the case of training, immersive experiences have proven to have high rates of learning retention, and, in the field, AR can be a useful communication tool between those on the ground and in a command center.

The Importance of Psychoacoustics

No matter how good motion controls or graphics within VR and AR get, true immersion cannot be achieved without paying attention to psychoacoustics. Psychoacoustics essentially defines how humans perceive sound, and the tenants of this study can be utilized in VR to maintain the masquerade of reality. Some people who may have never experienced mix reality before may be put off by the unnatural aspect of the experience. Life-life, object-based audio is key to helping users acclimate themselves to an XR environment. Audio acts as the door into interactivity, and the ability to localize directional audio helps users feel as if they have control over their environment. The longer an illusion of realism is maintained, thanks in part to object-based auditory cues, the longer a user remains in the headset so long as their immersion is uninterrupted.

Employee Training

©Embodied Labs

XR has become invaluable in training, especially when simulating situations that would be unsafe for people to be in or impossible to recreate in real life. In the Dima Lab project, VR is used to train caregivers for patients with dementia. The Dima Lab experience allows trainees to step into the shoes of their patients by projecting unto the interacting party hallucinatory episodes that a dementia patient may experience. That caregivers are given the opportunity to experience what their patients are seeing from a first-person point of view is something that would be impossible without VR and allows caregivers to empathize with their patients in new and informed ways.

In designing audio for this project, it was important to keep in mind the point of view of the user. Those who are in this experience are embodying a human with a medical condition rather than an omnipotent viewer, and it was important that the audio reflected that. So auditory cues were head locked and condition-based audio interactivity to accommodate hand tracking was implemented, allowing for an experience that one caregiver says changed the way he interacts with his patients.

In the Field

Search and Rescue 5G (SAR5G) is an AR application that utilizes 5G to support emergency relief teams on the ground and remotely increase communications and improve decision-making strategies with remote resources. In high-stress environments where visibility may be inhibited, high-quality audio is essential to assisting a team with navigation and awareness. Auditory cues within a user interface can be used to direct the wearer to a certain point, or notify them of any changes or dangers in their immediate environment.

An overview of the SAR5G application

There are many factors to take into consideration when designing audio for an application like SAR5G. Since the application is intended to be used on the ground, the noise level of the surrounding environment and possibility of visual impairment were a couple challenges that needed to be designed for. It is important to keep the application’s audio at a level above the environment’s noise floor. In action, the noise floor could be measured by onboard microphones and adjusted for on the fly with presets built into the application that adjust the mix as necessary. This is especially important levels of visibility are low as auditory cues become critical to locating fellow scouts, escape rooms, or victims, to name a few.


As mixed reality continues to grow, more uses for the medium will be discovered and it will become ever more essential to our daily lives. Without attention being paid to the auditory design of these applications, their effectiveness is lessened, whether it be in the level of immersion that can be achieved or the basic functionality of the program. With spatialized audio, XR can be taken to the next level.

For more information on the application of psychoacoustics in XR, feel free to reach out and look forward to a video of our talk at GameSoundCon.

Noctvrnal

Amanda Galemmo

Written by

Noctvrnal

Noctvrnal

NOCTVRNAL | An audio studio based in Los Angeles that specializes in spatial audio design for interactive experiences like virtual reality, location-based entertainment, and installations. We’re bringing immersive sound to the forefront of the new media movement, with the goals

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade