Engaging with the Senses through Data: Spotlight on Brian Foo
How Foo leverages multiple senses in bringing data “visualization” to the American Museum of Natural History
What comes to your mind when you hear the term “data visualization?” For most people, it may conjure up images of charts, plots, and maps. However, there exist other methods outside of the visual sense to access and explore data, such as through sound and touch. Folks such as Brian Foo specialize in expanding the scope of data interaction in ways that will be highlighted this week at Nightingale.
Brian Foo is a data artist and innovator. He has built up a portfolio of work on making information more accessible to the general public; building new ways for the public to interact with this data using visualization, sonification, interaction, and immersion.
For example, in The Sound of Movement, Brian encodes a US geographic map with video and sound from Black Lives Matter protests to fully immerse the audience. On the other hand, The Climate Change Coloring Book is a project that allows users to physically interact with the data, and in doing so, stimulates further thought on the subject.
Naturally, Brian’s various projects featuring senses like sight, touch, and sound are a great fit for Nightingale’s Data Sensification Week. I had the honor of interviewing him about his career and projects. The following transcript has been lightly edited for clarity.
Could you start by giving an introduction for the readers?
Brian Foo: Yes, I am a data visualization artist at the American Museum of Natural History in New York City and was most recently the Innovator in Residence at the Library of Congress. I have worked in libraries and museums for the past decade, so my work has more and more focused on making public resources such as audiovisual collections, scientific datasets, and cultural objects more visible and accessible to the general populace.
How did you get into the field of data visualization? What drew you to it?
BF: Data visualization was the natural “glue” between what were previously my separate worlds of computer science, design, and visual arts. I believe my aha moment came when I was tasked to visualize the nearly 200K digital items released into the public domain by the New York Public Library. I subsequently created an interactive map using data extracted from the Green Book collection, an incredible collection of travel guides published between 1937 and 1964, that listed hotels, restaurants, and gas stations where Black travelers would be welcome. That project really got me thinking about how I can use familiar visual languages such as maps and trip planners in unexpected ways. In this case, my goal was less to convey the raw data and more to design an experience that forces you to think about how the size of the world can change depending on the color of your skin.
Have you made any “standard” data visualization pieces? Why are you interested in visualizing through other senses?
BF: Much of my early non-visual data visualization (a phrase that admittedly takes a little getting used to) work developed from my dissatisfaction with traditional visual representations of data such as charts, graphs, and maps. I think this was the result of internal conflict between my identities as an artist and data scientist. From a data science/communication point of view, I firmly believe “standard” data visualizations are the most efficient way to convey large amounts of quantitative data, primarily because that’s what people know how to read. But as an artist, I am left dissatisfied by everything else that is missing. And what that is differs from dataset to dataset. It often involves thinking about how the viewer or listener should feel about the data. It’s hard to evoke a feeling from a basic chart or graph. This type of thinking is generally off limits to the data scientist who is trying as much as possible not to bias the viewer’s experience of the data. But this is the gray area that I actually love to work in. And it’s why I am drawn to this idea of “visualization” through other senses such as sound and touch.
How do you think visual media differs from projects featuring other senses?
BF: I usually like to talk about this through the medium of sound, or “sonification.” I think many people start to approach sonification through the lens of visualization. For example, you can imagine a simple line graph and then “play” the graph with sound using either pitch or volume over time. While this approach can be useful in some contexts (such as accessibility), it’s much less efficient and effective than simply showing a visual representation of a graph that people are already familiar with.
Instead, I like to start with thinking about the strengths of the specific medium. In the case of sound, you can think about why heart rate monitors and Geiger counters exist. Our ears are good at passively detecting subtle change. This is what led me to be fascinated with the idea of “data music.” For example, how can the contrasts of a song echo the dramatic differences of income in a city? From the perspective of an artist, music has a couple of other strengths: it can evoke a visceral feeling, and it gets stuck in your head. What if I can get you to be thinking about income inequality all day?
Exploring Sight
I found the Light Reminders project really interesting as it reminded me of a similar concept mentioned in I Am Not A Robot, a Korean drama. In the drama, the female protagonist invents a pair of heart shaped lamps. The lamps are meant to be shared between two people and lights up upon touch, so that the other owner can see and feel when you’re thinking of them.
Is this an ongoing project or has it concluded? I‘m curious about how your behavior of contacting your friends changed with COVID-19.
BF: I love simple yet poignant visualizations like that! Light Reminders came out of a similar sentiment rooted in how I can visualize (and improve) my interpersonal relationships. The idea is this really slow and subtle form of visualization where my living space is the medium. Each lightbulb in my apartment was replaced by a “smart” lightbulb and is associated with a particular friend. If I don’t see my friends for a while, my living space slowly gets darker and darker. If I come back from a party, everything becomes bright.
This has been an ongoing project, however, it only exists as an online visualization now since it’s not fair to my wife to have a dark apartment just because I have a lapse in seeing friends! COVID-19 has definitely impacted my behavior, but it also has followed the trend of seeing friends less often as we get older, and life tends to gets in the way.
Exploring Touch
I appreciate the thought process behind the Climate Change Coloring Book and think it acts as a great educational resource in two ways — it can educate the reader about the data but also gets the reader engaged and hopefully excited about data visualizations.
How did this project come about?
BF: This project was in response to my shock in learning about the gap between scientific consensus and public perception of climate change. The Climate Change Coloring Book focused on addressing this gap using a hands-on approach — literally having people “color in” the data. I really liked how the concept of the coloring book is this nice mixture of casual and active. As adults, we typically consume data in tiny morsels of charts and graphs. This coloring book forces you to sit with (and reflect on) climate data for potentially hours. I also find it compelling (and perhaps fitting) that your hand starts to hurt right as you attempt to fill in the data about fossil fuel carbon emissions.
You mention that the dinnerware produced through the American China project is meant to be functional. Have you been using them in daily life?
BF: Yes, I have used the pieces in daily life, but more importantly I hope to pass it down to future generations to trigger conversations about our family, and Chinese American history in general. I really like the tactile interaction with the tea pot, which is supposed to communicate the weight and temperature of the traditional iron used in Chinese American laundries (and my own family laundry, as it were) for decades.
Exploring Sound
I found Data Driven DJ very interesting and really appreciate your detailed write-ups on how it was created, so thank you for that.
BF: Thanks — I like to think that my documentation is one of my “signatures” as an artist. Especially as a data visualization artist, I find it important to be transparent about my process. My representations may not always be perfect, so I encourage others to understand my thinking and perhaps even improve upon it. I also benefit greatly from being able to look back at documentation of projects from a long time ago. Sometimes they give me clues for where to go next!
I’m curious if you’ve studied music or music theory before?
BF: I have not studied music or music theory before, which is kind of what made my Data-Driven DJ project so fun for me. Each song was a learning opportunity to study a new musical concept or genre that fits well with the topic. This is also why the documentation is so detailed — in some cases, the thought process ended up being more compelling than the end result!
I am inviting the public early into my design process because I feel strongly about being able to respond to your feedback in a meaningful and substantial way. Whether you are a musician, student, researcher, or curious citizen, I want to make sure these tools and resources are as useful to you as possible.
In your guest post on the Library of Congress’ blog, you mention that you’re inviting the public early into your design process for Citizen DJ. Did you want to try out a new process specifically for this project, or do you try to incorporate feedback within all of your projects?
BF: Anything I do that is a public tool or product on behalf of an organization, I attempt to incorporate feedback from as diverse of an audience as possible. I think that’s vital for any product design process. However, much of my personal creative projects don’t have that burden since I classify those as personal “art” rather than products. That said, accessibility is a central goal to all my work, professional or personal, so I try to always keep my ears open for feedback.
As evidenced from your detailed documentation, it’s clear that so much thought and research goes into each project. What’s your research process like? Would you say that it’s the most time consuming part?
BF: All of my projects typically start with me wanting to learn more about something, whether it’s climate science, how to make music, or exploring my family history. The research phase is usually the longest and most open-ended part of any project and can take months or even years. Often the challenge is thinking about how to match the topic with the right medium or vice versa, which is an organic process that can come suddenly, gradually, or even never. Sometimes the medium comes first, and sometimes the topic comes first.
Through your projects, you’ve experimented with various tools. Are there specific tools and programming languages you prefer over others?
BF: I lean towards the most popular open source tools and programming languages like Python and JavaScript. This typically gives me the most publicly available materials and resources to work with. It’s also important to me for other people to be able to access, run, and improve my code. This is why I try to focus on using public data as well, rather than proprietary data. And all of my projects are open source by default.
Is education and open source accessibility something you’re passionate about?
BF: Yes — this comes from some combination of my work in libraries and museums as well as my experience as a programmer who heavily relies on open source software. I believe the restriction or suppression of information and other forms of knowledge creation is at the root of a lot of our society’s problems. Conversely, open and democratic access to such data and resources is fundamentally part of the solutions to those problems.
What’s next for you? Do you have any upcoming projects you’d like to share?
BF: There are a couple of projects I’m working on now that have me thinking about the manual and imperfect nature of data collection and how we as data artists and designers can communicate that. For example, I am working on a National Monument Audit with an organization called Monument Lab where the goal is to assess and visualize the current landscape of monuments across the United States. I think there might be a broad assumption or desire for the resulting dataset of monuments to be comprehensive and complete. This is in fact a near-impossible task, especially when you dig into the question, “What is a monument?” Like with my past work, it will be critical to document the data collection and visualization process. However, here I’d also like to explore ways to make that process apparent in the visualizations of the data itself so as to potentially expose what’s missing or incomplete.
Thank you for your time, Brian! You’ve gained a new fan and I’m excited to “see” and experience your future projects.
Jennifer Li is a Digital Analyst based in Toronto, Canada. Outside of work, she loves exploring data visualizations as well as learning new languages.
