Sound Color Project

Exploring accessibility to sound through color, light, and texture.

TL;DR — just visit Sound Color Project.

I’ve been a musician for as long as I can remember, and a designer for almost as long. I grew up listening to Kris Kross and Crystal Method. I also grew up making spinner paint art and drawing Dragon Ball Z characters. Music and visual art have always evoked feeling for me. As a professional designer and hobbyist musician, sound and color continue to feed my spirit every day.

In 2019, I began experiencing ringing in my ears and what I thought could be progression of hearing loss I was diagnosed with after an illness in my infancy. I became nervous, wondering if I would completely lose my hearing, and how I would connect to music. I could still play my instruments, be in bass, drums, or guitar, and feel vibrations if frequencies were low and volume high. But how would I experience the feeling of the music, the mood conveyed through a pattern of chords or notes?

I decided to go for a hearing test to actually get some medical advice on what could be happening with my hearing. To my absolute surprise, doctor told me I didn’t have any hearing loss at all, that my hearing was actually really good. They said that the ringing and occasional moments of hearing loss could be related to nerve damage in my jaw from a surgery I had in adulthood. I feel very thankful that I am able to hear well, and understand the privilege I have to even go to the doctors to get tested, and to be able to hear, speak, and communicate in that way.

This experience alarmed me, and during the time I was under the assumption that my hearing was impaired, I was doing a lot of thinking around if I were to lose my hearing one day, how could I experience that feeling of music I mentioned before. Is there another sense I could tap into that can convey an array of emotion? I tapped back into the artist/designer side of me and thought, maybe this is possible using the visual sense.

So I began to research different ways sound could connect to color. I found a variety of studies that have been done, and began to take notes. Some of those original studies included:

• Chromesthesia (sound-to-color synesthesia)
• Chakras (frequencies of energy in the body)
• Chromotherapy (healing through color and light)
• Emotion (relating a wheel of emotions to genres of music)
• Adolescence (learning sound and color in early developmental stages)

I’m continuing to learn about new ways to connect the audible and visual spectrums, but these studies helped me get started. You can read more about how these studies are used in Sound Color Project on the website.

After studying different materials and articles on these topics, I started creating different color palettes for each of them based on examples provided in public studies, reference materials, or my own interpretations. It has been important to recognize that while some of these studies are founded in science, there is not a science to the art of relating sound to color in every instance. It’s also crucial to understand that we all see color differently, depending on our own photoreceptors, the environment we’re in, and the lighting that is provided. We all perceive color differently, if we are gifted with privilege of sight. I was actually diagnosed with a form of Red-Green Color Blindness, and you can read about my experience as a partially colorblind designer in an article I wrote.

Since color is simply an illusion of pigment mixed with light, our ability to see color is depending on light. Therefore, the color of the light source, or what the light is bouncing off of and reflecting on what we determine the color of, has an extreme influence on what we determine that color is. For example, if you turn on a white light in a room with white walls, the walls will appear white. But if you’re wearing a pink shirt and you move closer to the wall, depending on how reflective the material of the white wall is, the closer you get, the more pink you will begin to see on the wall. Now is the wall still white, or is it pink in that moment? Artist, Olafur Eliasson has done a lot of work around these ideas, which you can watch in season 2 episode 1 of Abstract in Netflix.

This idea that those of us who see color perceive it differently, and my conversation with a friend a musician who has Chromesthesia, led to want to also enable people to create their own palettes of what note would relate to what color.

But how would all of this be actualized?

One of my colleagues and friends had experience working with the Web Audio API and I just happened to tell them about the project and vision. We worked together to create a website that uses the device microphone (with consent of course) to analyze the audio it is hearing and, in real time, translate the audible frequency to a color on the page. We have experimented with how to convert audible attributes to visual ones, like the volume to the brightness of the color.

As of right now, as I’m writing this, we are preparing to present the project at Sound Scene 2020, put on by the DC Listening Lounge, and originally scheduled to take place at the Hirshhorn Museum in Washington, DC. We hope to learn how this technology could be used in the D/deaf community as we talk more with accessibility experts and people who are D/deaf or hard of hearing. We’ll continue to explore the different ways to relate sound to color, and might even dabble in exploring the opposite — how we can turn a visual image into an audible representation.

Though this article may tell a bit of the story, the best way to understand this technology is to experience it.

Visit the Sound Color Project

Please feel free to reach out to me at derektorsani@gmail.com with any questions or ideas you may have for this project.