Same Data, Multiple Perspectives: Curse of Expertise in Visual Data Communication
TLDR: Visualizations can communicate ambiguous messages. What we see in a visualization may be guided by our background knowledge. Therefore, it is worth annotating data to communicate one specific message or explore depicting data in a new way to see patterns with fresh eyes.
Ambiguity is everywhere.
For example, what animal is it on top?? A duck? Perhaps a rabbit? Depending on whether you see the two strip shapes on the left as rabbit ears or duck beak, you come to completely different conclusions.
A visualization is like a collection of these rabbit-duck illusions. While one rabbit-duck figure can be interpreted in two ways, a graph offers up many ways to see them. This leads to ambiguity. Two people looking at the same visualization could pay attention to different patterns and draw opposing conclusions. Let’s see this in action.
Say a small East European city introduced legislation banning private citizens from carrying concealed handguns in public, in hopes of reducing violent crime rates. Data analysts put together a visualization showing violent crime rates in the four years following the introduction.
Is the legislation effective?
As shown in Figure 2, people who hold differing beliefs about the relationship between gun ownership and the number of violent crimes may focus on different aspects of the visualization.
Can two people really look at the same visualization, and see different things?
Furthermore, when people see a certain pattern in a visualization and draw a conclusion, would they think everyone else would see the same pattern and draw the same conclusion from the visualization?
My colleagues and I demonstrated this case of visualizations as ambiguous figures in the lab. We invited people into the lab to learn about an intriguing election story.
What do you see as the most visually salient pattern on this visualization?
You may point out that there is drama between the Alliance and the United party — the two lines crossed each other twice.
And yes, there is indeed drama between them. The Alliance party has been leading the election throughout, until an initial debate in May. The leader of the Alliance party insulted the spouse of the leader of the United party, which made him look cruel in the eyes of the public, losing him a good chunk of public support. The United party used that opportunity to catch up. At a later debate, however, the United party leader couldn’t come up with a good answer to a debate question, so he tried to bring back the old topic of the spouse insult. This made him appear very unprofessional as if he was avoiding real issues. As a result, his support dropped and the Alliance party took the lead once again.
Now that you know the drama, what do you think someone who is not reading this article with you, who didn’t get to learn about this story, say your neighbor or your boss, what do you think they will see as the most visually salient feature on this visualization?
You are likely thinking, maybe they would see the bottom intersections of the two darker lines as the most visually salient.
Or not. You may be thinking of something else. But do you think that person would see this other pattern — that the two green lines (Labour and Alliance) are symmetrical, mirror images of each other?
Now you may be thinking, that’s not fair. The intersection of the bottom two darker lines are much easier to see than the mirror image of the two green lines. So had I told you a story about the green lines — the Labour and Alliance party — instead, would you have noticed them being mirror images of each other?
Here is the story. There is drama between them too.
The Labour party has been leading the election throughout, until an initial debate in May. The leader of the Labour party insulted the spouse of the leader of the Alliance party, which made him looked cruel in the eyes of the public, losing him a good chunk of public support. The Alliance party used that opportunity to catch up. At a later debate, however, the Alliance party leader couldn’t come up with a good answer to a debate question, so he tried to bring back the old topic of the spouse insult. This made him appear very unprofessional as if he was avoiding real issues. As a result, his support dropped and the Labour party took the lead once again.
Cool. Now is the mirroring of the green lines easier to see?
And you are probably thinking, wait a second, I just told you the same story. That’s cheating.
Well, sure. I told you the same story but fitted it to a different pattern on the visualization. This made the pattern of the green lines mirroring each now stand out to you. Do you think the other person you imagined (who is not here with you) would see the mirroring green lines as more visually salient now?
You can think of this as the rabbit perspective of looking at this visualization, and the previous bottom intersection perspective as the duck perspective of looking at the visualization. By learning about the drama behind the pattern, you saw the respective pattern as more visually salient and predicted other people to do the same.
Still not convinced?
Sure. Imagine your neighbor or your boss again. How likely do you think they would have noticed that the top two lines — the Labour and the Conservatives — also had drama?
Do I need to tell you about their drama?
You likely realized that the pattern of the top two lighter colored line can also fit the same story about the two debates I just told you. You can think of this as a third perspective of this visualization.
After people read one of three versions of this story and became experts on this visualization depicting the patterns mentioned in the story, they reported what they found the most visually salient pattern on the visualization. They also predicted what other people, who are uninformed observers with no knowledge of the story, would notice first from the graph.
It turns out, depending on the version of the story they heard, participants rated the patterns mentioned in their versions to be the most visually salient on the graph. They even predicted that uninformed others would see the same patterns mentioned in their versions to be the most salient, even though they know that these uninformed people have never heard of the story!
In a way, the participants are ‘cursed’ by their knowledge, no longer able to see the visualization from an unbiased point of view. Call it the curse of knowing too much, expertise can make it hard for us to take someone else’s perspective.
You see this ‘curse’ in action everywhere.
Business partners often report that, in monthly budget meetings, the data analysts excitedly discusses a pattern in a visualization, but few people in the audience saw that pattern. Most are thinking of the visualization as “literally inconceivable” (but they might nod along anyway).
This happens because the expert knows the materials they are talking about very well, but struggles to separate their expertise from that of their audience, as it is very hard to ‘turn off’ what they know. This ignorance of experts provides fertile ground for miscommunication, leading to poor decisions by people relying on visualized data.
This curse can also coincide with motivated reasoning, which is a decision-making phenomenon in which people make decisions based on their beliefs rather than evaluating the evidence.
For example, when shown a global temperature change visualization, those who believe in global warming may see the pattern of rising global temperatures and conclude that there is, in fact, global warming. Those who do not believe in global warming may see that the global temperature fluctuates but stays relatively flat, concluding that there is not enough evidence suggesting global warming, just as they thought. Both sides are convinced that everyone else should be seeing what they see (You can read more about this here).
In short, you can be cursed by your own knowledge.
We must keep in mind that visualizations can be ambiguous, and what seems so salient to us may not be the only truth. Of course, becoming aware is only the beginning.
How can we avoid this curse?
First, the design of a visualization influences what comparisons are made, or what conclusions are drawn. It is worth to keep exploring data visualization techniques to express data in ways that can thwart this curse of knowledge bias. For example, people are more likely to compare proximal values, so one simple change could be to rearrange the values in the same visualization to see it with fresh eyes. Sorting values or hitting the ‘swap rows and columns’ button are great places to start.
Second, because the curse of knowledge is tough to detect and inhibit on your own, getting other perspectives through other people’s critique or feedback could help. Invite your friends (or enemies) to look at your visualization. They can share with you what they understood from it and what they found confusing. Rinse and repeat until you are satisfied (or until when the visualization is due).
Third, this curse of knowledge shows that a visualization can be interpreted via multiple perspectives. If multiple people merge their multiple perspectives and interpretations, they can gain a more complete, less biased understanding of the data.
As I finish writing this, I worry that the more I understand the curse of knowledge, the more I will be cursed with the knowledge of the curse of knowledge, confusing you as I try to explain the curse of knowledge.
Reference
Xiong, C., van Weelden, L., & Franconeri, S. (2019). The Curse of Knowledge in Data Visualizations. IEEE Transactions on Visualization and Computer Graphics. DOI: 10.1109/TVCG.2019.2917689
