Visualization
Non-Designers & Data
Visualization today has changed the role of the designer from a beautifier to a knowledge producer — but this was not always the case. When data visualization began to emerge in the 18th century, its pioneers weren’t designers by profession. Some of the earliest visualizations were done by engineers, social reformers, and physicians. Information design, in this era, was born out of the necessity to deal with the amount of data being produced during the Industrial Revolution and the need for graphics to support ideas, clarify problems and propose solutions.
Unlike much graphic design, the purpose of visualization is insight — not pictures. Though often graphic in nature, information design based on data can introduce new concepts and make powerful arguments. Just through the process of building visualizations alone we are able to solve problems. John Snow‘s infamous map of London’s 1854 cholera outbreak is a perfect example of how process lead to the knowledge gain that changed how we understand the spread of bacteria and the importance of data visualization. Snow meticulously collected data on each outbreak of cholera in the London area. After plotting the outbreaks geographically, the source of the disease, the Broad Street pump, was easily identified and removed. Though Snow collected data and wrote an extensive statistical analysis on the breakouts, the map was a language anyone could understand. This map not only ended the 1854 outbreak, but was also the answer doctors and scientists sought for how cholera spread.
Visualizations after their creation also play a role as an important communication tool. They can act as a support or reinforcement for an argument, which expresses something that we already know but need to reveal in a meaningful way. In this manner, data visualization has the unique potential to clearly convey complex ideas and provide a way to “reason about quantitative information.”
Digital Developments
In the last 150 years, we’ve developed extensive principles for collecting and displaying data. With the advancements in digital technology, we are now able to create and filter through organized digital structures of data in a way that was earlier restricted by the visual capacity of paper. In the 90’s with the invention of the World Wide Web, there was an enormous amount of data that became suddenly available. It was natural that information design became increasingly important as a remedy for the need to condense and communicate. As defined in Using Vision to Think, “Information Visualization is the use of computer-supported, interactive, visual representations of abstract data to amplify cognition.” When we build tools for users to explore their own data, we should assume that people want to connect and engage. Information visualization further facilitates that human interaction by using a graphical language. When it takes the form of a digital tool, users are able to manipulate data in real-time. This encourages visual calculation and results in the detection of patterns. We find satisfaction in sense-making and finding correspondence between events separated in time and space. When we interact with designs that enhance dimensionality and clarity of content, we are able to participate in discovery. Analysis becomes not a banal or mathematical act, but a creative one.
Today phrases like “big data” and “information overload” are buzzwords. By the second we collect and produce an immense amount of raw data. But all of this data is only useful when we derive insight from it. In the last thirty years, we’ve been using data as both a reason to turn inward on ourselves and as reasoning for future decisions.
Telling Our Own Narratives
As we discussed in the recording chapter, abstraction tools like a tally system and then clay tokens were key when quantifiable data became too big to memorize. Proto-writing, the category in which mnemonic, pictographic and ideographic languages exist, has been used since prehistory to tell narratives (Figure 13). These visual languages were used to express esoteric ideas, make predictions of the coming day, and help lead the dead through the afterlife. These pictorial languages gave a visual identity to each member of a society and therefore each person could become realized through language. The power of imagery to tell stories and create identities was the first method used to communicate and engage before we constrained our interactions by an alphabet.
In his Information Visualization Manifesto, Manuel Lima writes, “Human beings love stories and storytelling is one of the most successful and powerful ways to learn, discover and disseminate information.” Like the ancient tribes and civilizations, we too have narratives, and plenty of data to support them. Not only do human beings love stories and storytelling, we especially love telling stories about ourselves. The reflective and expressive writings discussed in the recording chapter were some of the first examples recording our lives as an act of introspection. The compulsive self-documentation of Buckminster Fuller and Thomas Jefferson’s meticulous observations on the weather (Figure 14) can also be seen as using manually collected data points to tell stories important to us.
It was once true that there existed no common conventions designed to depict one’s life as a series of daily rises and falls, but the introduction of data-capturing technology into our daily lives changed that. Our own personal pocket data collectors laid the groundwork for the increased obsession of telling micro-stories through self-quantification. It is no wonder that the popularization of data visualization happened at the same time as the introduction of the smart phone. It gave us the ability to not only capture what we discern to be important or relevant, but also to capture anything without placing severe demands on our sense of credulity. In the early 80’s, Steve Mann created the first general-purpose wearable computer allowing for continuous live first-person video to be captured. Wearables are becoming more popular but also more integrated with our favorite device, the mobile phone. Today there are countless quantification apps optimized for mobile use. They track the foods we eat, the exercise we do, how we spend our money, how we sleep and how productive we are. And that just scratches the surface. To find a sense of identity in these terms is tedious — but no longer impossible.
Many of the applications that collect quantitative data are visualized as a change in one variable over time. Typically, the data collected or displayed isn‘t sensitive or expressive by nature, therefore pie and ring charts, line and bar graphs and points on a map are effective. The social elements of quantified-self applications are undoubtedly a useful feature to motivate users. We are astute observers of other people. The success of social media tells us that we love to see what others have and do. However when we choose to tell our stories through steps, bank transactions or food intake, we are first observing ourselves. We are gaining knowledge on our own behaviors and inevitably learning more about our own identities. We can take these types of personal data and add it to the knowledge we have internally of how we have acted in certain situations, how other people react to us, and how we feel about it all. These are the things that often cannot be displayed in a chart.
In order to express the full richness of a story, self-quantification begs for the support of a richer context in which to view it. The Feltron Annual Reports from Nicholas Felton are one of the most recognizable examples of self-quantification in the field of data visualization. In the Feltron Annual Report for 2013, Felton designed his data from a communications perspective in light of the Snowden NSA leaks and subsequent prevalence of information privacy in public debate. “My work is driven by an interest in storytelling, facilitated by core concerns of graphic design…and a tenuous ability to use Processing to organize data into compelling constructions.”
Because Felton’s data is personal, there is no way of verifying its accuracy or that of the visual language he uses. Together, the visualizations in the reports act as a comprehensive narrative of Nicholas Felton’s year. Communication data is difficult to quantify, but it illuminates truths and re-contextualizes his experiences. The reader can try to draw conclusions about who he talks to most and through which medium he most often communicates. This type of information visualization attempts to show the ephemeral and is interested in bringing understanding to the relationship between quantitative and qualitative. The Feltron Reports contain data which can be quantified and though stylized, can be displayed in standard visualization patterns. This technique has satisfied Felton’s project for over ten years. But what else is it outside of an overt display of “data sexualism”? A collection of quantifiable data…just because? Such a public exposure to personal data begs the question, why not go one step further? Why not tell us how you feel about who you talk to the most and why. The why’s reveal Nick Felton to us, not the quantities. And who he is, is arguably more innately fascinating to us.
Data visualization needs to move past methodologies that encourage a one-size-fits all approach to visualization because each entry, bar or number is an individual story.
Even the tally mark of the first transaction represents the interaction and exchange between two people. Data is sometimes slow, blurry and too complex for objective inspection. As we are the sole experiencers of our own lives, we shouldn’t expect machines to tell us something revealing about ourselves.
Mental Health & Mood Visualization
An earlier use of the term “visualize” meant an attempt to construct an image in the mind as a way of using mental processing to initiate real-life health improvements. As I’ve discussed, technology has given us a more concrete, data supported approach to visualizing. In 2016, psychologist and emotion researcher Paul Ekman conducted a survey of emotion researchers to gather data on what they all agree about.
His findings, combined with an idea from no other than the Dalai Lama was the basis for Stamen’s project, Atlas of Emotion. The project utilizes a fairly loose definition of “atlas” but still gives a viewer the possibility to “explore the landscape of emotions,” find out where they come from, and how we react to them. This map does what most maps do, in that it helps the viewer gain a little more knowledge about a space where they are either in or affected by. Yet it is bound by what all emotion researchers agree on. Thus yielding a very thin amount of information in a very general context. A map about the human experience is as interesting to us to the extent of how empathetic we are. What might be more engaging, would be putting emotion into a context matters more to us: ourselves. We care deeply about ourselves and how we feel. It’s why we write about it and publish it. If we want there to be a wider conversation about emotion, we need a digital experience which would allow information sources to merge. One where the input is the things that are personally relevant to us and the output is information about how the things that affect us as individual persons relate to the theme‘s bigger picture. We have the technology to capture our experiences and filter our lives, let’s use it to take a more personalized approach.
An earlier use of the term “visualize” meant an attempt to construct an image in the mind as a way of using mental processing to initiate real-life health improvements. As I’ve discussed, technology has given us a more concrete, data supported approach to visualizing. In 2016, psychologist and emotion researcher Paul Ekman conducted a survey of emotion researchers to gather data on what they all agree about. His findings, combined with an idea from no other than the Dalai Lama was the basis for Stamen’s project, Atlas of Emotion. The project utilizes a fairly loose definition of “atlas” but still gives a viewer the possibility to “explore the landscape of emotions,” find out where they come from, and how we react to them. This map does what most maps do, in that it helps the viewer gain a little more knowledge about a space where they are either in or affected by. Yet it is bound by what all emotion researchers agree on. Thus yielding a very thin amount of information in a very general context. A map about the human experience is as interesting to us to the extent of how empathetic we are. What might be more engaging, would be putting emotion into a context matters more to us: ourselves. We care deeply about ourselves and how we feel. It’s why we write about it and publish it. If we want there to be a wider conversation about emotion, we need a digital experience which would allow information sources to merge. One where the input is the things that are personally relevant to us and the output is information about how the things that affect us as individual persons relate to the theme‘s bigger picture. We have the technology to capture our experiences and filter our lives, let’s use it to take a more personalized approach.
In the mainstreaming of quantifying personal data, mental and physical health applications have also appeared. MedHelp’s web-based mood tracker allows users to record a data for a number of physical and mental health categories. Users are able to include diagnostic information, write journal entries and interact with other users via the MedHelp community forums. The interface is extensive and complicated but allows for a high detail overview of a user’s daily experiences including mood heatmaps by the day and line charts for fluctuation in mood. Mood tracking mobile applications are also not in short supply on both iOS or Android app stores. What can be observed in almost all of them is the encouragement of daily mood or emotion tracking, journaling and some type of visual output. Visualizations range from emoticons and calendars to traditional patterns like line and bar charts, pie charts, heat maps, and word clouds.
What all of these applications lack is abstraction. There is an argument for the use of emoticons, as they are often used in therapy and counselling settings to help patients identify emotions when they lack a deep vocabulary. Outside of this example, there are no expressive characteristics to the methods of visualizations which would reflect the actual experience of emotion, feeling or mood. Instead, these attempts fall back on imposing boundaries for emotion labeling and confining interpretation to x- and y-axes. Gary Wolf, writer and co-founder of the Quantified Self movement wrote an article for the The New York Times Magazine in 2010 called The Data-Driven Life. In the article he writes how valuable the act of quantification is, “Numbers make problems less resonant emotionally but more tractable intellectually.” He goes on to praise electronic trackers for their ability to be “emotionally neutral” simply reflecting back to us our values and judgements. But should this be celebrated? Should we minimize the human elements of ourselves in order to achieve a sense of finite and completeness — as if we could be summed up in a few simple charts. By doing this we embolden the idea that everything can be understood and everything can be explained through causality. This of course, is categorically not true.
Expressive & Experiential Visualization
At this point we know that emotions are hard to measure and communicate, which inevitably makes them difficult to quantify and visualize. Even if we are able to identify the emotion label for an experiences, we would be hard-pressed to determine every possible stimulus that would trigger it. What we linguistically identify as the same emotion can be rooted in different sources and manifest itself in different ways. For example, sadness can be rooted in neglect, when we are feeling left out or alienated. It could also be rooted in sympathy, when we are confronted with the hardships of others. This is one of the many reasons why our experiences cannot be rated and compared evenly across one arbitrary value scale. This proves to be quite the challenge in using data visualization as a method of mood and emotion expression. Often the “how” and “why,” the qualitative data, is critical in these cases. In order to approach these types of datasets, we need a new set of guidelines.
For centuries we used written or spoken language as a means to express our thoughts and feelings. Psychologists have established CBT as the gold standard, problem-solving approach to discussing emotion experiences. Using it, we focus on the way we perceive things and how they makes us feel. But our experiences are still elusive, still intangible, and we are sometimes unable to find the right words. We are restrained by our language. By using visualization, we can see things in a new way. Because when we put things into graphic form, we liberate further potential for action. Nevertheless, we designers are also bound. We rely on the ability to quantify information before we can construct a useful visualization. If we could find a way to mix these two languages, that of numbers and concepts with that of shape, color and pattern, then we could really begin to visually depict “softer” data.
Data that is more than just about facts. Data that is often ambiguous. This type of visualization must embrace inexact interpretation. Interpretation as a creative construction as opposed to a precise reconstruction.
The process will have to be flexible with gaps that may never be filled and events that may never be explained. The focus would need to be on providing a lexicon for both the one experiencing an emotion and the one who wishes to understand their experience.
An interesting example of this type of visualization is Dear Data, a project by information designers Giorgia Lupi and Stefanie Posavec. Lupi and Posavec spent 2015 recording their personal data on postcards and shipping them across the Atlantic. For each week, a topic was chosen (like instances of indecision or frequency and type of complaints) as well as a system with which to visualize the data. Some of the systems reflect traditional patterns but others are more expressive. Both because of the project’s hand drawn quality and its allowance for imprecise representation, a visual dialect is constructed that conveys more than numbers and categories. Ambiguous line lengths and colors not constrained by accessibility give a freedom of expression to these instances of very personal experiences. The process of recording personal data consistently, as we’ve seen from Felton, Lupi and Posavec, is tedious, but in doing it, one becomes more aware of themselves and their surroundings. The Feltron Annual Reports and Dear Data are compelling in their methods of recording data and their methods of visual expression. Both encourage self-awareness, perhaps without meaning to, and both have defined graphic styles which have since been copied and modified by many designers and non-designers alike.
Dear Data is graphical thinking. It is the process of recognition and naming, together, through drawing. Drawing personal data is a method of mediation between perception and reflection. Yet still from these examples there is no visual language which can be systematically applied. What is true in the therapeutic setting is also true in expressive visualization: the person attempting to express themself has their own personal criteria by which they perceive their experiences. If an expresser poorly presents their experiences, the receiver has little chance to translate the language back into the experience (Figure 15). Only the most skilful author can translate an experience and craft a story (or image) well enough that any listener or reader could truly understand. Such a craftsman would need both a keen awareness and broad vocabulary. Even so, this is a challenge where “language is characteristically helpless.” When we speak about emotion, we speak about something that, by definition, is indescribable. Both of the expresser and receiver need multiple exchanges before a common language and criteria can be established between them.
The case may be that we will always be forced to fit our emotions into words, to try and quantify them or to retreat altogether to methods more abstract, like art therapy. We might always be bound to “make do” languages. But at one time, hazards like floods and droughts were unimaginable to those who had not experienced them, so we represented them with pictograms. At one time, our memories weren’t enough to retain large quantities and many transactions, so we made ledgers. Today, the amount of data we have is simply too large for the eye to detect patterns, so we build charts to support our decisions and linear models to predict our future. Yet in spite of the breadth of languages we’ve developed — linguistic, statistical, and visual — we remain handicapped in the expression of something so innate. One of the only things that we all, as humans, share.
Continue to Fine: The Application.
An overview of this project and a link to the log book of my process can be found online here.
