How to Make Information Graphics With A Piano
In recent years I’ve moved my design career from animation and interaction design to data visualization. While my prior work experience certainly helped in the transition, I found that what has always shaped my visual work the most is the discipline of classical piano.
When I’m exploring a data set and working through hypotheses of how to visually encode and express the information, it’s nearly always the structure, craft, and art of classical piano interpretation and performance that inform my process. I’ve always played the piano so I don’t know what it’s like to do visual design work without this perspective, but I do know it helps.
Playing music trains us to see and feel patterns that loop on small and large scales; to love foundational technique work and harmonic analysis; to listen more critically to how others play pieces we’ve learned; to think in terms of our listening audience and how they will experience the piece; to craft a presentation of the work that will engage people; and, perhaps most importantly, to tell a story that brings the audience on a journey from beginning to end.
The Data of Music
Musical notation is a language and data format that encodes, at the most basic level, which notes to sound, and when. Each dot or circle represents one note, arranged vertically for pitch (low to high) and horizontally (left to right) for time. Several other symbols and notations are employed to indicate rests (breaks or quiet moments), phrasing (which notes belong together in a continuous idea), accidentals (modifying notes outside their key signature), indications for tempo, volume, style and many more nuances of performance. At its core, scores like the one above are a map for where to put your fingers on the keyboard, and when.
If you rotate the score 90 degrees clockwise, you get:
While music can only be experienced in real-time by listening or playing, its score can be read, analyzed and understood by studying it atemporally, like any other text. With the entire score at hand, we can cross-reference different parts out of sequence and think about the larger structure. When we take into account the era in which the composer wrote the piece, contextualize the musical culture in which it was composed and examine the nods and references to past or contemporaneous work, the implicit limitations of their instruments and cultural milieu, and the bold risks they took. A composed score is a complete data set that we can survey, to better understand it and to inform how we’ll interpret it for our listeners.
The Music of Data
In data visualization we often start with an idea, question or hunch, with a goal of finding an answer and conveying it to a particular audience. Sooner or later we have to find a dataset to engage with and begin the discovery work of uncovering patterns and signals. Here’s a typical, by no means complete, process:
- Generate question/hypothesis, including audience: who is this for?
- Seek and find data: wrangle it into a useable form.
- Research prior examples: what analytical or design work has been done before in this space? Where are there opportunities to expand or explore?
- Basic data analysis: seeing the shape of the data, does it look like what I thought, or anything interesting?
- Sketch ideas: build around a story or focal point.
- Work more with the data: see if it can support or refute the story idea
- Broaden your scope: look for possible complementary data sets to add
- Iterate through visual representations: what marks or shapes should encode the information?
- Test with your target audience as well as peer designers: does this make sense? What could be improved?
- Subtract: find elements to remove or hide, in order to clarify your main theme
- Refine, Polish, Publish
I approach “performing” a dataset with these rough analogues in the back of my head:
I saw Tony Chu give an excellent talk at the OpenViz conference a few years ago where he discussed the many design decisions involved in creating r2d3. He quoted designer Barbara DeWilde:
Graphic design is the use of space to control time
Meaning that the design choices we make can control the experience and attention of the viewer. This can refer to design fundamentals like hierarchy, white space and gestalt, as well as to more sophisticated and combinatorial use of design technique and composition.
We can invert this quote for music:
Musical composition and performance is the use of time to control space
Let’s assume the listener is situated in one place and focuses on a performance that moves through time. It’s the sequence of notes in melodies and motifs, the layers of notes that form harmonies, the change of harmonies over time, the movement from conflict (dissonance) to resolution (consonance), and the mood and texture of the piece that change the listener’s experience of being in the space they occupy.
Judging Intangibles
Piano works are parallel timeseries of multidimensional values underpinned by an invisible harmonic architecture. As we hear and experience the piece being played, we’re also unconsciously seeking patterns and tracking what’s happening over the entire piece. Like a novel or movie, musical works exist sequentially in time and tend to start somewhere specific, go to interesting places, create tension, return to familiar themes, develop subplots, and ultimately return to the initial “home” theme or idea.
How do we evaluate a piano performance? With my teachers, it’s along the lines of:
👉 Is the performer telling a story or just hitting all the notes?
👉 What decisions did they make to call out certain phrases and downplay others?
👉 Is the overall tone and feel appropriate to the period and the composition?
👉 Does the performer understand the historic context and harmonic (structural) patterns in the piece? Are they expressing those?
👉 Is the performer taking any risks?
👉 How does the performance guide the listener through the piece?
👉 Where could the performer improve technically?
👉 Where could they improve expressively?
👉 As a listener, did you feel like you went somewhere and experienced something?
And we can easily translate these questions into the evaluation of information design:
👉 Did the designer craft a narrative or are they just displaying all the data?
👉 What choices did they make in selecting which parts to focus on and which to leave as background?
👉 Is the overall tone of the graphic appropriate to the data and topic it’s showing?
👉 Does the designer understand the provenance, distribution and statistical significance of the data, and are they expressing or explaining those?
👉 Is the designer taking any risks?
👉 How does the visualization lead the viewer through the information so that it flows and it’s comprehensible?
👉 Where could the designer improve visually or technically?
👉 Where could they improve expressively or stylistically?
👉 Did you feel like you learned something and you would remember or share this graphic?
It’s definitely possible to push the analogy too far — the main difference being that piano scores are works of art intended to make cultural statements and evoke feelings, while visualizations are designs we’re using to teach, compare or evaluate information for a specific task. But data visualization spans a spectrum from data, science and accuracy to experience, emotion and art. With both music and visualization I try to alternate looking at the big picture and get clear about exactly what I’m trying to express, while also working through the atomic details to ensure my craft and technique supports the expressive goals at smaller and larger scales.
I’ll close with some condensed excerpts from an interview with Yo-Yo Ma on the Song Exploder podcast. The host, Hrishikesh Hirway, asks Ma to compare his own performances of J.S. Bach’s cello suites from 1983 and 2018. When listening to his older recording Ma observes:
“Very good cello playing, the notes are even, everything is just very measured — very competent… This person cares about having a nice sound. This person likes to make things look good.
But this music, it starts our imagination going: where is he taking us? And then I see, well: there’s this stop, in the middle. Has that person thought about that great interruption? Does that person hear the pedal point that’s in there? I’d say: maybe subliminally a little bit? But not something that is front and center. So you get someone’s priorities when you listen… you get someone’s value system.
This music, the structure of it is totally clear. So in order to bring this to life, you actually have to breathe life into it. And that pause, it’s something that’s totally, violently unusual. It screams out, saying ‘something happened!’”
When asked about his 2018 performance however, he noted:
“There’s more attention to changing landscape. There’s less emphasis on, say, ‘let’s make a beautiful sound,’ and there’s different kinds of texture. There’s greater fragility. There’s more attention to the bits of landscape that says, hmm, wait, look at that. Check that out. And so what does all this mean?
Like a great book that you read several times during your life, each time you read it, it’s the same book. But you certainly get very, very different material from the same story: ‘Oh, I didn’t see that, I didn’t notice that before. What is that?’ So there’s an evolutionary process.
Which means that any experience that you’ve had has to be somehow revealed in the process of making music. And I think that almost forces you to make yourself vulnerable to whatever is there to be vulnerable to. Because that, actually, is your strength.”
