How Two Musicians Transformed New York into an Audio Augmented Reality Concert Experience.

From Central Park to Lincoln Center, step into a secret soundworld of original compositions.

“Being able to understand someone else’s story is what really drives this piece. I hope that listeners are encouraged to be more curious in their lives, whatever that means for them.”

-Lucy Yao

Lucy Yao and Dorothy Chan, Chromic Duo, are constantly dreaming up innovative ways to reinvent how we experience music and storytelling. The duo blends classical music, toy piano, and electronics into genre fluid performances and installations, constantly blurring the lines between film, VR, and AR. Their current work, Emerald Futures, puts special emphasis on the wonder of the everyday, reflection, and empathy.

For this audio AR experience, Chromic Duo teamed up with the New York Philharmonic Very Young Composers program, an afterschool program that exposes students to the instruments of the orchestra and culminates with original works performed by professional musicians including members of the Philharmonic, to feature original compositions during the sound walk.

Reflecting on the AR sound walk, the artists spoke to us about what went into creating Emerald Futures:

How did you begin working with the New York Philharmonic Very Young Composers program?

Yao: One of the teachers from the program reached out to us because we’ve been in these hybrid performance spaces. We decided early on that we wanted this to expand upon what we’re already doing as musicians which is to communicate a message and be community facilitators. Why do we play in the first place? How do we want people to feel?

This was an interesting opportunity for us to ponder how to make this performance really memorable, especially for someone who is a promising young composer. We know what Zoom fatigue is, and we didn’t want to go down the route of Zoom performances or recording a video. There’s nothing wrong with that! But we wanted something more. How could we take the surroundings of New York City and create something interactive that can exist on its own without us needing to be there in real time? With Gesso, it’s so cool that anybody can go in and experience this throughout the summer and hear audio and visuals being triggered together. It’s almost like you’re putting yourself in the middle of a movie and you’re having the music evolve around you.

Chan: When we came across Gesso, we knew we wanted to make something special, something more participatory for the young composers to display their work. This kind of work can be adaptable to a concert space whereas it’s also enjoyable as an audio AR experience through the app.

How did you choose the route for the sound walk?

Chan: We thought planning the walking route around the Lincoln Center area would be nice especially for those participating in this project. They would know the area better and could better imagine what the final sound walk would be like. We wanted to reflect upon these ideas of transformation and transitioning. There’s a lot of progress and we wanted this route to reflect that. We decided to start in Central Park, walk towards the city, and go towards Lincoln Center. There’s a progression, moving from one place to another.

Yao: I don’t want to go back to the way things were. There is awakening now, and I hope we continue to think about creating a more equitable and diverse society. I think this transition of going into Central Park gives the listeners the ability to reflect within themselves. What are the things you had to reckon with during the pandemic? As the walking route goes back into the city, it’s like ok now that you’ve reflected within yourself, what can you do now within your community so that you’re creating a future you want to believe in? This is how we came to the title, Emerald Futures. It’s so great that the young composers are able to facilitate that transition as we’re reflecting both within ourselves and our community.

Can you expand on the importance of the last stop being Amanda Phingbodhipakkiya’s “We Belong Here” mural?

Yao: It’s amazing when you find a piece of work that when you’re confronted with it, it completely pulls the brakes. For me, I had that feeling when I saw Amanda’s mural. When I was growing up, I never really saw anyone looking like me in Hollywood or popular culture and I didn’t realize that deficit until the pandemic. It made me question this absence that I didn’t even know was missing. So when I saw the mural, it was hard to ignore and felt like this was a really cool time to be here and recognize this, that our stories as Asian Americans matter. I think the one commonality Dorothy and I discovered when talking to Third Culture Kids is just how much Asian Americans are not really sure what it means to belong. We grow up between one culture and another so we’re trying to create meaning and belonging. I hope the last piece on the tour creates that feeling that your story and your being can exist without being tied to one identity.

Do you have any favorite moments from the walk or production process?

Chan: Education is at the forefront of what we do. The process of us working with the young composers was really memorable. We all weren’t in the same space, I think some of the kids weren’t even in the United States anymore at that point, and yet they came forward with so many ideas and you could see how they’re inspired by different things in their life.

Yao: The whole walk is special but one part that was really special to me was where you can see a clear example of how music shapes the visual world we’re in. At one of the stops on the sound walk, we talk about joy and play. We think about going back in time and having the playground be a way that you can relate music that’s reminiscent of childhood.

Chan: There’s that one spot where if you listen, the music will actually be accompanied by children playing in real life. You can actually hear kids playing in the playground with the music from the walk and it’s such a powerful moment when it’s paired together.

What advice do you have for other composers who might be interested in working with location-based audio?

Chan: Go on a walk! When we put on headphones (especially the noise cancelling ones!) when we’re blasting through a playlist in some place, that becomes really amazing. Working through that, think about how you can curate this sound walk experience where it’s a specific setting, but not too specific because we don’t actually control what happens in a particular space at any given moment. It opens up a space of imagination, so listen and imagine what journey you can create with this tool.

Yao: Immerse yourself without music and see what happens. See if you can start listening to your environment, and it doesn’t even have to be a nature walk. You could be in the middle of the city in a chaotic area and just go there and observe what’s happening. Take a phone and record yourself in that environment and use free tools to start messing around with these sounds. Think about the person walking through the space, what they might hear and imagine, and the story starts to develop.

What do you hope listeners will take away from this experience?

Yao: I want people to walk away with a feeling that they can slow down and look at things around them and find a new way to understand how someone else might think. Empathy is a big part of this. Being able to understand someone else’s story is what really drives this piece. I hope that listeners are encouraged to be more curious in their lives, whatever that means for them.

Chan: There are many moments when we ask the listener to listen, and in our lives, listening is one of those things where it’s easy to say and hard to implement. To reflect on that and allow space for that is important.

Emerald Futures is available to experience for free on the Gesso app, and be sure to share your experiences with the artists using #VYCxChromic and #EmeraldFutures on Instagram.

Gesso is a platform for storytellers to turn their ideas into immersive, audio experiences. If you’re a composer, performer, audio producer, or other creative professional interested in working with us to create location-based augmented reality sound walks, join our creator platform to get started.