Sound Translation — Collaborative Visualizing

Samantha Rauch
Feb 1 · 6 min read

For February 3, 2021

I chose to translate the “ding” our toaster oven makes when the timer runs down. It’s quite a sharp and piercing noise, taking over your whole hearing for the first second or two, but it mellows out into a sweet ringing that resonates until it’s inaudible, which takes about ten seconds. The noise lives at the very top of your hearing — you hear it fully in your ears rather than in your body — and it feels very smooth and almost symmetrical.

I played with shapes that I felt captured that experience. I played with color for some of them, using colors I felt captured the lightness of the noise (the blue capturing the metallic-ness of the noise and the yellow capturing the sweetness). For all iterations, I tried to capture the resonance of the noise in some way, making some more of a right-to-left reading experience and making others more like what I felt the shape of the noise itself was.

For February 8, 2021

We received our sound assignments. Shannon, Thien and I are working on the fourth music excerpt, which I’m quite excited about. The first half of the sound sounds like something from a noir detective movie, with a stand-up bass lurking in the background as a full glass of drums vibrates until it pops, splashing shimmering cymbals over the chords of an electric keyboard. Then, as the noise fades out, you get hit in the head with a barrage of echoing trumpet that’s the auditory equivalent of when people throw superballs as hard as they can inside locker rooms and watch them pinball off the walls, backed by a discordant jumble of keyboard and bass as the sounds they make tumble like a piano in a clothes dryer. Finally, there’s one last shrill headshot of trumpet that carries from one side of your hearing to the other, followed by a few low rumbles on the stand-up bass.

I did a few interpretations that focused on the listening experience of the piece as a whole as well as some that focused only on the instruments.

My notes on the listening experience of the song and some ink interpretations of the piece as a whole.
My interpretations of the individual instruments, some more focused on properties (what I’m defining as the quantifiable aspects) and others more focused on qualities (what I’m defining as the emotional experience of listening).

I focused on all of the interpretations being a line that you read left to right. The piece we have is long and the front and back halves are very distinct from one another, so I didn’t want to do the piece a disservice by compressing both parts into a visual format that might not properly convey the duality of the excerpt, especially since it’s part of what makes our excerpt so distinctive.

For February 10, 2021

I wanted to hone in more this time on A) how all the sounds interacted with each other, B) how to show the sounds moving through space and time, and C) representing the sound as reconstructed memory (I wasn’t listening to the clip every second I was drawing, I was drawing it from my memory, and as Emily pointed out in class, our brains remember only the most important parts and fill in the rest).

I also decided to lean into my natural tendency to gravitate towards graphical representations, even in the abstract, rather than working against it.

In the first piece I did, I allowed the flow of the piece to be linear, as if I were writing down a weird pictorial version of sheet music meant to be read left to right as a way of expressing space. For the second two pieces, I tried to focus a bit more on my gut reaction to the music while also incorporating elements of what I heard.

For February 15

Thien, Shannon and I all compared the pieces we did to come up with a list of criteria by seeing what we had done similarly — how many sections of the song we felt there were, what were the most important things to record, how the sound should flow in 3D space, the mediums associated with certain instruments, etc.

One of the first things we figured out is how we wanted the sound to move in 3D space — we knew we wanted to have it start close to the viewer, move away, and then come back again. This also worked with another criterion we came up with quickly — that the song had two distinct parts.

With a bit of discussion, we came up with our criteria:

  • the trumpet section is sharp and scribbly, made with dry media primarily
  • wet media should be used to communicate echoes and reverberation
  • the beginning is rounded and faded, at a contrast to the high-contrast and angular second part of the song
  • the sound moves through 3D space as shown above

We also, critically, opted to prioritize memory over reality — instead of trying to make “sheet music” where we recorded each instrument, we acknowledged that some instruments work together as one sound in certain parts and instead focused on capturing the piece as it that sticks in your memory, emphasizing the important identifying characteristics and de-emphasizing the rest of the noise.

With that, and a bit of photoshopping to combine sections, we came up with this:

For February 17, 2021

We talked amongst ourselves as a group about the piece we made as well as asked another group for critique. The general consensus between us was that the form works well in space and in communicating the sound, but that some of the depth is lost in the lack of contrast and that the trumpet charcoal is a little too crazy for the discernable pattern that the trumpets actually make.

Using that information, we made a new trumpet section to overlay on top of the old one (decreasing the opacity on the old one) to get a more methodical sound, but we kept the old one in place to serve as the reverberations/echoes created by the trumpets and as the discordant background keyboard and bass that’s happening underneath them.

We also adjusted our contrast to better show 3D space, since we realized we were using darkness to indicate proximity to the viewer. We darkened up some parts, like the keyboard spots in the ink beginning, and lightened others, like the trumpets and the cymbal shimmer at the end of the first section, to move them closer and farther away from the viewer, respectively.

The old version
The new version

We also did a bit of smudging of the cymbal shimmers to help better integrate them into the first part, since they felt like a new section rather than a continuation of one.

Finally, we increased the size on the trumpet blast ink blot and made it a bit darker black to really help bring it out to the viewer.

Overall, I’m quite happy with it, and I think Thien and Shannon are, too. We feel the form moves very organically with the sound and has the sense of depth that we’ve been after.

Sam Rauch Collab Viz

Projects from collab viz, spring 2021