What Does Data Sound Like?

Acoustic Infographics: Making sense of complex information with sound

Photo illustration by Emilie Barbier, used via creative commons.

We are used to seeing data visualizations, but how might audio helps us get a different sense of the data? Are there ways our ears can help us understand data better than our eyes? Indeed, scientists at NASA suggest that by converting data into sound they may be able to scan vast data sets in a matter of minutes.

Last week I asked Jody Avirgan, FiveThirtyEight’s podcast host, “What does data sound like?”

Avirgan pointed to one of his stories: “Can You Taste The Algorithm In This Arugula?” in which he explored “how farms big and small are using data to control sunlight, nutrients, air circulation and more.” In the video he looks out over a field of greens and asks “Where is the data?” Getting an interviewee to point out how data manifests itself in the phisical world — the sound of shades closing automatically to control temperatures, or sprinklers turngin on in tresposen to sensors — is one way give voice to the data.

Sound and data are not so far apart and there have been some really interesting attempts by newsrooms and artists to turn data into songs and sounds. I call these acoustic infographics.

U.S. Home Prices, Sung As Opera : NPR

NPR’s Planet Money team turned to opera to illustrate the Case-Shiller home price index. “You can see the boom and bust all in one simple graph,” write Jacob Goldstein and David Kestenbaum. “But when we go on the radio to talk about home prices, a graph isn’t much good to us — nobody can see it. So we converted the Case-Shiller graph into musical notes.”

An Olympic Musical: New York Times

Kevin Quealy (a graphics editor at the New York Times) has a great Tumblr of charts and infographics, which is where I found a few audio infographics related to the Olympics. The NYT graphics team and Amanda Cox worked map the finishes of close races with sound. This was based on a project Cox did for the 2010 winter Olympics.

Above is Amanda Cox’s original audio infographic and below is one of the new videos she made with the New York Times for the London games. There are many more versions and videos over at the Kevin Quealy’s Tumblr.

Using Brain Wave Data to Convert a Seizure to Song: Brian Foo

There has been a lot of discussion about virtual reality and empathy, but artist Brian Foo believes that turning data into songs can also spark empathy. He writes about his project, Rhapsody in Grey, “This song generates a musical sequence using EEG brain wave data of an anonymous epilepsy patient. It examines the periods before, during, and after a seizure. The goal is to give the listener an empathetic and intuitive understanding of the brain’s neural activity during a seizure.”

Hearing Colors: Radiolab

It’s no surprise that RadioLab has toyed with the form. In the segment below they use a choir to represent the light spectrum, specifically that of a rainbow. The voices in the choir represent the various colors, and the hosts explore how different animals see a rainbow based on what colors they can see. However, being that this is radio, we get to hear how the rainbow would sound, rather than see it.

Listening to Radioactive Decay: The Radioactive Orchestra

Jihii Jolly pointed me to this examples. She writes in her post about the infographic, that visualizing how energy moves is difficult. Enter the Radioactive Orchestra. “The Radioactive Orchestra was developed into an online application for anybody, where users can pick up to five isotope frequencies, place them on virtual mixing board and edit their pitch and tempo. A visual music video of sorts is also automatically played to the left of the mixing board. These beats can then be saved and shared.”

Singing The Euro Zone Crisis: Marketplace

The complicated financial system and recent debt crisis in the European Union is tough enough to explain in print — how do you do it on radio? Marketplace from American Public Media decided to use a Barbershop Quartet.

Algorithms as Audiblization

The person who created the video below calls it an “audibilization.” Here is how he describes it: “This audibilization is just one of many ways to generate sound from running sorting algorithms. Here on every comparison of two numbers (elements) I play (mixing) sin waves with frequencies modulated by values of these numbers. There are quite a few parameters that may drastically change resulting sound.”


Follow Josh Stearns on Twitter or subscribe to the Local Fix for a weekly dose of innovation and community engagement for newsrooms. Originally published at storify.com in 2014, updated in 2016.


Thoughts on Media is a community publication, curated by ReadThisThing. Follow us on Medium or check out our newsletter.