Audio-Spatial Storytelling
What if the true potential of augmented reality is not visual, but audio-spatial?
For the past few decades digital storytelling have been delivered mainly through screens. The networking of the image, whether pictorial, written or moving have largely expanded the canvas of media makers and helped flatten the media horizon. The centralized audio-visual medium of television has been challenged by the proliferation of distributed networked screens, multiplying the perspectives and voices allowed to be heard. But while screens were emancipated from the single cords that fed them, the messages they convey are still trapped within the boundaries of the screen. Standard or wide, low or high-res, one-way or interactive, these screens have been cropping our horizon into predefined blocks. This limitation is surfacing more and more these days with mobile computing hitting the mainstream and bringing with it the hype around “Augmented Reality”. AR is an emerging technological trend layering perceived reality with networked channels to add, contextualize and augment our bare senses. As with digital media, the prime sense to augment has been first and foremost our sight. But the experience of augmented sight has been mostly underwhelming. It seems we are not that excited about moving in space and having to fixate our view through a 5 inch smartphone screen or even through a Retina Display tablet. And so, even though we’re seeing a lot of development in this field in recent years, we are yet to see an audio-visual augmented reality experience dramatically augmenting our lives.
But visuals are not the only expressive medium to undergo a revolution with the rewiring of the media space. Sound has been a huge part of the equation. Networked audio has brought the records industry to its knees and have developed new media topologies in the forms of the iTunes store, streaming radio, Podcasting and Skype. Yet, I would argue we’ve only been scratching the surface. The potential of networked audio is still largely untapped. With sight being the most primary sense, it is understandable that when attempting to augment reality we would first turn to visuals. But when it comes to the production and dissemination of culture, sound has actually been networking for much longer and much stronger than images. After more than 100,000 years we are wired to communicate through vocal language. Oral culture has helped frame the human experience much before the invention of writing and in a much wider scope than image making. Possibly more than anything, sound is spatial, it surrounds us, it does not ask us to be its spectators, it simply absorbs us within it.
Sound is tied to location, many of us have probably experienced the phenomenon of audio/spatial Synesthesia. This is a strange experience of our brain pairing different senses together. For example, walking in a certain street, listening to a certain song can pair that song with the spatial experience of walking in that street. Listening to that song could bring up memories of that street. And vice versa, walking in that street can bring up an imaginary soundtrack in our heads, playing the memory of the song to augment our spatial experience. That is the experience I first felt when we started running the You Are Not Here project (youarenothere.org).
In the summer of 2006 I was a part of a group who designed You Are Not Here, “a dislocative tourism agency”. YANH meta-tourists used a two sided map and a telephony system to explore Baghdad walking through the streets of New York and later to explore Gaza walking through the streets of Tel Aviv. The map overlaid the paired cities encouraging the participants for example, to navigate the Baghdad/NYC map to the West Village to “visit” the Baghdad Zoo. The remote location was explored through audio tours, augmenting the city you’re in with the narrative, the audio and the (imagined) spatial experience of the remote place. Years after I could still find my way around Baghdad, walking in the streets of Williamsburg. New York has become a platform for stories of a place far away and it was that very audio-spatial experience that made the stories stick.
It is audio, not images that are a perfect fit for what could be a networked experience with a true potential to augment reality. YANH was an early experiment, back in 2006 it was using a printed map, a telephony system and stickers in the streets. Today, audio spatial experiences can be self-contained within mobile applications, taking advantage of digital mapping, audio streaming and geolocation technologies.
These days, podcasts are becoming more popular and the audiobook medium is on the rise. We’re getting more comfortable with customizing our audio fix to our current preference, we might also enjoy customizing it to our current location.
Beyond an invitation to explore this emerging creative opportunity, I would argue that when exploring the new networked media possibilities digital storytellers have mostly been looking under the streetlight. We’ve been networking the written word, networking the moving image, networking the audio recording, but each medium has for the most part been maintaining its original dynamics and characteristics. It is the merging of different media that is truly worthy of the problematic title “New Media”. Audio-spatial is a new networked approach that is based on the much older networks of oral culture and on the origins of storytelling. If you’re looking to augment reality, you might want to start there.
The article was originally written for the 10th issue of Shoppinghour Magazine (UK) titled Feast on Listen.
