How the Apple Vision Pro Will Shift the Audio Experience: Part 2
Ready to jump back into the deep waters of spatial audio? If you have read the first part of my exploration into the audio solutions of the Apple Vision Pro, you will recall how we dived into the profound impact of spatial sound on our virtual experiences. We tapped into the tech Apple is about to release and scratched the surface of how sound sculpts our very reality. If you missed that ride, catch up here.
As we step into Part 2, let us go beyond the surface. We are about to venture into some details of audio raytracing, and object-oriented audio, and take a glimpse of the implications and chances opening up for engineers.
Audio Raytracing: The New Frontier in Sonic Experience
Raytracing, in its essence, is a technique initially found in the world of 3D visualizations. By tracing the path of rays (or beams) of light, visual raytracing can simulate how light interacts with objects, reflecting, refracting, and casting shadows to create realistic images. This same approach of tracking and simulating interactions has been applied to audio, allowing for a far more lifelike sound experience.
Instead of light rays, think of sound waves emanating from a source and interacting with the environment. Like light bounces off surfaces and refracts through transparent materials, sound waves reflect off walls, get absorbed by soft material, and diffract around obstacles. Audio raytracing captures these interactions to create a sonic landscape that reflects the complexities of the real world.
Let us ground this in a practical scenario: Imagine wearing the Vision Pro and still seeing your physical surroundings together with the VR windows of Vision OS. Your actual room, filled with various furniture and objects, each made of different materials, confined by walls, naturally shapes every sound. And I am not just talking about being able to locate the sounds; they resonate in harmony with what you visually perceive in your surroundings.
But what about the Vision OS sounds emerging from its speakers?
Like the Vision OS graphics seamlessly blend into the real world, so does the sound. This realism is achieved thanks to the integrated LiDAR scanner and TrueDepth camera of the Vision Pro, which automatically maps the 3D layout of your environment. This means that spatial sounds are not arbitrarily placed within your physical space; they resonate as if they are part of your physical surroundings. And as far as I know, this will also work in pitch-black rooms, with the tracking sensors using infrared. What a time to be alive. 🚀
Approaching the Spatial Audio Landscape
So, spatial audio is not just about delivering sound. It is about constructing an immersive soundscape that mirrors reality. Being not only limited to left and right but above, below, and all points in between. All of this happens with two outputs, each at one ear.
The concept of surround music isn’t a recent idea. Over the years, numerous albums across different genres have been crafted for a surround sound experience. Notably, The Alan Parsons Project’s “Eye In The Sky” received a 2017 Surround version, and Depeche Mode’s iconic “Violator” album is another fine example. While these masterpieces were engineered with 5.1 speaker systems in mind, to immerse a room in multidimensional sound, the contemporary challenge lies in translating this expansive sound to headphones, which are conventionally limited to left and right channels (more about what this means later).
Apple, with artists like Billie Eilish and Elton John, is nowadays diving deep into Spatial Audio, though only a fraction of Apple Music’s library currently embraces it. Devices like the AirPods Pro, AirPods Max, and the 2023 HomePod highlight Apple’s dedication. Beyond Apple, Tidal and Sony’s 360 Reality Audio is making waves, with Samsung’s Galaxy Buds 2 Pro and Google’s Pixel 6 also joining in. Gaming isn’t far behind, with gear like the HyperX Cloud Alpha.
However, the audio engineering part has changed significantly, especially looking at object-oriented sound. This is one of the most fun parts of working with Spatial Audio and the Apple Vision Pro.
5.1 Surround vs. Object-Oriented Sound: The Future of Audio
5.1 Surround Sound has been the gold standard in home and commercial theaters for years. It employs six discrete channels: front left, center, front right, rear right, rear left, and a subwoofer, enveloping listeners in a hemisphere of sound. Here, sounds are assigned to specific channels, creating a directional experience.
In Object-Oriented Sound, each sound becomes a free entity or object, moving within a 3D space without being tied to any channel. Instead of being bound to specific directions, sounds can dynamically travel around, above, or approach the listener.
What does this shift mean for the industry and consumers?
For Audio Engineers: Object-oriented sound offers an unparalleled canvas of creativity. Engineers aren’t restricted by channel limits, allowing for intricate soundscapes that adapt dynamically to user interactions or scene changes. Undoubtedly, this is a more intricate process. But the reward is flexible sound sources adapting to your surroundings and visuals.
For Consumers: The immersion is intensified manifold. Whether in gaming, VR experiences, or movies, the sound isn’t just around you; it’s alive, moving, and matching what you see in your VR space. So, matching immersive audio to your visual experience.
The Evolution Continues
As we bring our little deep dive into spatial audio to a close, understanding that this is not just a technological advancement but could mean a paradigm shift is essential. The Apple Vision Pro, with its embrace of audio raytracing and spatial sound, exemplifies how the digital and physical worlds are getting closer than ever before.
For VR users, the promise is clear: a future where you not only see apps and OS windows as part of the room but the sound matches, too. It’s not just about better tech but about creating more intimate, realistic, and evocative experiences that blur the line between the digital and the physical world.
With every technological leap, there are bound to be skeptics and early adopters. But it’s undeniable that spatial audio, especially realized in devices like the Apple Vision Pro, challenges our notions of what’s possible in sound. As the tech evolves, so will our understanding and appreciation of its potential. As with all innovations, there is a damn exciting road ahead. ⭐
