Process: Exploring the Process of Immersive Audio Stories

Following the journalistic process

Published in

The Stories Within

7 min readMay 27, 2020

This post is a part of The Stories Within series, a project by the students of Parsons School of Design as a part of the Spatial Computing Challenge under the guidance of the R&D team at The New York Times and NYC Media Lab. The project is introduced in The Future of News Series!

“How might we design an experience that highlights the emotional connection and unheard stories knit in a local community?”

Our goal was to focus on a newsworthy story that gives a timely account of a significant event or development within the local communities.

For our case study, we focused on the Meatpacking District as we found out that there was an interesting dynamic shift from food to fashion industry and we wanted to depict the personal narratives and perspectives of locals through spatial audio.

About the Meatpacking District

It is a 24-hour neighborhood located on the far west side of Manhattan, NY, bordered by Chelsea to the north and the West Village to the south. In 1886, the city declared the area as a public market to ensure that they participated in the profits. At the turn of the last century, the buildings that were once dwellings, stores, and warehouses were quickly transformed into meat businesses — the vestiges that we still see today. In the past few years, many of the meat businesses have relocated to the Bronx, but a variety of new businesses have replaced them and adapted these remarkable, historic structures once again — so that the neighborhood retains vibrant, 24-hour energy that defines Gansevoort as unique in New York City. This has created an incredibly interesting and diverse dynamic that is fitting, given all of the inherent contrasts of the area.

Finding Sources

Our preliminary research enabled us to have a basic understanding of who the prominent figures were and their relationship with the neighborhood. We started compiling our list starting from independent organizations and public historians who had done significant research on the Meatpacking District. We thoroughly went through all the available news stories already written on the subjects before interviewing anyone about it. We reached out to them in order to schedule an interview and locate the appropriate places to set the mood for the interview. Then we prepared a list of detailed and open-ended questions to get the needed information.

Our research involved going to the site and finding local stories. As we started exploring the neighborhood, we came across several stores that had gone through drastic changes in the last couple of decades. But the one store that caught our eye was the store Madewell since it had transitioned from a French diner to a fashion boutique, but still held the signboard of “R&L Restaurant”. Here’s a summary of what we encapsulated from our meeting with the founders:

The Story

Almost 25 years ago, a young Frenchman named Florent Morellet, took over a diner named the R & L on Gansevoort Street. Eventually, it became the hotspot of the neighborhood. People started coming over from different corners of the city and it became a huge hit. Over the years since, the Meatpacking District transitioned from food to a fashion hotspot. In 2008, Florent’s store also transformed into a fashion boutique named “Madewell”. When we visited the store and started exploring the space, we found an interesting connect between the two stores. Shaunda Robbins, the current manager of the fashion store and an avid preserver of the legacy of Florent Morellet, had done an exceptional job of preserving the nostalgia of the old restaurant. In order to keep up with the culture, Shaunda has preserved the aesthetics of the old counter and some other nuances of the restaurant how it used to be earlier. She also has safeguarded laminated copies of several certificates and articles about the restaurant so that she could train her employees with the culture that’s associated with Morellet’s establishment. During our conversation with her, she kept insisting that Morellet’s legacy was one of the most powerful stories that outshines the culture of the Meatpacking District. After our interview, when we started listening to the conversation between these two, there was a blissful spark that teleported us to a different dimension. In that moment, we realized that this was a local community story that we must bring to light.

Once we figured out we’re going to bring this deeply rooted narrative to the spotlight, we classified our research process into two phases:

We thoroughly started analyzing the existing content-rich materials and digital archives, and then obtained relevant information about the cultural significance and evolving milieu of the neighborhood.
We then drafted a list of people who can provide the most accurate information about the neighborhood and how to contact them

Collecting Assets

We interviewed a number of people who were closely associated with the Meatpacking District and documented the entire process in the form of stereo sounds. For all our interviews, we used Zoom H6 recorders clubbed with wireless microphone kits. We initiated the conversation with a list of prepared questions, plus other questions that come up in the course of the conversation. We persistently asked all our subjects to suggest other sources in order to gain additional leads for our story and we also made sure we maintained a good rapport with all of them.

As the Interview process branched out to second and third sources, we ended up doing follow-up questions and extensive research which eventually helped us to connect the missing dots and gain a solid understanding of the evolution of the neighborhood.

Since our whole experience is solely focused on audio elements, we made sure to record all the relevant ambiance so funds that introduce and familiarize the users to get a better understanding of the setup and tone of the narrative.

Let’s talk about the technical aspects of audio recording.

When it comes to the recording process itself, the biggest consideration is what kind of results we’re hoping to achieve, which is directly related to how much editing we want (or have time) to do. If we want to produce fully polished recordings for a spatial audio experience, then we’re looking at a complex multitrack layer. Instead of just monitoring through our gear during soundcheck to see if we’ve got adequate sound and good levels, we did playback of our recording so we can identify the missing elements in our recordings before it’s too late.

Organizing

We constantly kept on reminding ourselves, “What’s the story we want to portray?” and “What’s the point in sharing this?”. These questions helped us to shape up our thought process and we had a clear objective before we started curating the crust of the narrative.

Organizing our audio and other visual materials is crucial for any design and storytelling process. And doing this in the right order effectively helped us with the process of archiving and later retrieval much easier.

We started archiving each story by designating a google folder just for it and then we started devising episodic audio clips in accordance with the structure of our narrative.

To kick things off, we named all our raw recording sessions by date and event/subject (if for a project, with a 3-letter project code instead of an event or subject), and we never erased or over-wrote our raw originals. We then did have many separate subfolders for edited sounds by type, and we leveraged Google drive for searching and cross-referencing audio clips via the specific filing convention we did for each and every clip.

i.e. “Meatpacking District/Day 1/ Location 1 /Person 1 Take_1.aiff”

Inside of each folder, we included everything unique to this episode, for instance, Ambiance Sounds, Interviews, Sound effects, Background Music, Audio Assets, among others.

A note about content labelling: some editing programs allow you to create a “label track” that can hold annotations at points along a specific timeline. Some audio recorders, such as the Zoom H6 (what we used), allowed us to create a mark at a certain time in our recording. More advanced apps and software like Audition can read these marks like chapters or bookmarks. Unfortunately, Audacity doesn’t support this.

Throughout our whole process, We constantly tried to blur the lines between objective storytelling and physical interactions by emphasizing the idea of an immersive spatial audio experience.After collecting and organising the audio assets, we started the curation process by drafting a script for voiceover in order to visualize how all the elements fall in the right place and it paved way to the production phase.

About the Team

Akshansh Chaudhary is an XR Experience Designer. In his projects, he focuses on social and world issues like privacy and local news. His design approach is to create immersive and playful experiences to spread awareness. Follow his work on akshansh.net.

Karen Asmar is a design technologist working at the intersection of the built environment, society and human-computer interaction. Her work focuses on exploring the impact of emerging technologies on ways we interact in space, with space and with data in space. Follow her work on karenasmar.com.

Ponvishal Chidambaranathan is a fervent immersive storyteller and digital producer with a strong inclination towards innovative, philosophically charged content in unconventional storytelling and interactive media. Follow his work on ponvishal.com.

Yashwanth Iragattapu is a creative technologist and interaction designer. He creates products that encompasses human and spatial interactions through emerging technologies like Augmented reality and Virtual reality. Follow his work on yashwanthiragattapu.com.

Debra Anderson is an entrepreneur and educator specialized in XR. She is recognized for designing data-driven and research based approaches to immersive experiences with a focus on how data and emerging technologies can be used for civic engagement and social impact.