Ten Tips for Developing Audio-Only Augmented Reality Experiences
I’ve recently attended a workshop for Bose AR’s new AR glasses and Wearable SDK. Given my background in music and that I’m learning how to design for XR, audio is a ripe vertical for innovation. Here are some tips and resources for when you find yourself designing an audio-only AR experience.
- Consider latency when looking at hearing-aid type applications. There’s a reason you can’t use any synth or drum machine on your iPhone over wireless headphones — the half-second audio delay ruins real-time processing.
- Check out the FB360 Spatial Audio Workstation.
- Use Soundover to integrate devices into group-based experiences. Their tech claims to connect any devices that have both a speaker and a microphone. You can theoretically triangulate user position, or transmit real-time data across devices.
- Acquaint yourself with two unique group audio experiences, the silent disco and the Decentralized Dance Party.
- Diegetic vs Non-Diegetic audio cues. In cinema, a diegetic sound is one that is contained within the universe of the story; a gunshot, some dialogue, or a band at a bar. Non-diegetic sounds include sounds in the film external to the story, such as a narrator or opening credits music. Consider reading Diegetic Cues for Guiding The Viewer in Cinematic Virtual Reality and inject some Hollywood into your audio app.
- Instruction vs Environmental (Music/Sound) User Guidance: will you tell somebody to turn left with the computer’s voice or guide them to look that way with an interesting or pleasing notification sound?
- Hands-Free Experiences — in the case of Bose’s AR Glasses, you are equally designing for a hands-free experience in addition to your audio-only software. Consider the uses in baking (having somebody walk through a recipe while cooking), surgery (allowing a remote doctor to guide a surgery halfway around the world), and podcasting (quietly and discreetly listen to audio content while working) and then consider the hands-free aspects of your experience.
- Is your phone the master, is the wearable the slave? or Vice Versa? Will your audio experience require interaction with the smartphone powering it, or will it be kept in the user’s pocket?
- Spatial audio is gimmicky — turning your head to hear a sound flow and ebb in volume is boring. Instead, consider the 360 audio aspects of your app as a freedom to take advantage of — rather than the user guiding the accelerometers and gyros to guide the sound, use the accelerometers and gyros to guide the user to experience the spatial sound.
- Consider Tunity to sync your experience with other new media content, such as a video being watched, a song being heard, or a TV channel. Acquaint yourself with the best practices for building new media second screen experiences.