“Effectiveness of Animation in Trend Visualization,” ten years later
At the IEEE VIS conference 2018, I was a member of a team awarded the Infovis “Test of Time” award for the paper, “Effectiveness of Animation in Trend Animation.” In that paper, carried out with a team at Microsoft Research, we tried to better understand how users interact with moving dot interactions, like Hans Rosling’s Gapminder talk.
Hi, I’m Danyel Fisher. I’m Principal Designer Researcher for Honeycomb.io, a system observability startup. On behalf of the team, I’m honored by this award.
To give some background on this work, I want to bring you back to 2006. I was working at Microsoft Research.
Hans Rosling’s “Gapminder” TED talk had gone viral: he compellingly showed international trends in public health and economics, with scatterplots changing over time. His excited treatment was inspiring. Everyone wanted a visualization of flying dots — and wanted to find a way to capture that excitement. Bill Gates’ office approached Research to ask what it would take to get a time-varying scatterplot into Excel.
Our team — Roland Fernandez, George Robertson, and Bongshin Lee — got together to figure out what our research approach could be. John Stasko was visiting us for a sabbatical, and he jumped in to help. Technically, implementing the animation wasn’t hard — Roland got together a working version quickly — but could users understand it?
We wanted to figure out precisely what was so compelling about Rosling’s talk. This was one of those wonderful cases where any result was interesting: maybe we’d be able to capture that magic — or show why we couldn’t.
We designed two alternative views. The first we called a “traces” view, which simply got rid of the animation in favor of drawing every trend at once on the screen — in the last few years, people have started calling that a “connected scatterplot.”
The second view showed a small multiples view. One of our most important design decisions was focusing on one small multiple per trend. Some later animation papers have shown one multiple per timestep; unsurprisingly, they all have discovered it’s really hard to follow individual items as they jump around the screen between frames.
We tested both an interactive version, and a version meant to simulate a presentation.
It wasn’t obvious what to test. We wrestled with questions about whether we wanted to reproduce Rosling’s talk, or to pick new datasets.
There were some places where our test was different from a Gapminder presentation. Each specific question referred to one or several countries that had direct, decidable answers on the graph.
Our results found that animation is a paradox.
It certainly wasn’t very effective for solving our tasks. Whether they had interactivity or not, users in the animation condition were less accurate than small multiples. Users took a very long time to answer questions when given an interactive animation, scrolling back and forth. When we took that control away, they’d quickly make their best guess. Despite that, interactive animation wasn’t much more accurate then non-interactive!
But we also found that users really liked the animation view: Study participants described it as “fun”, “exciting”, and even “emotionally touching.” At the same time, though, some participants found it confusing: “the dots flew everywhere.”
This is a dilemma. Do we make users happy, or do we help them be effective? After the novelty effect wears off, will we all wake up with an animation hangover and just want our graphs to stay still so we can read them?
Either way, this sort of animation has appeared in professional tools. Researchers have built on our structure for other visualization types. (Looking back, I do regret that we didn’t release raw data from the user study.)
I think our paper helped drive part of the current conversation about the differences between presenting data and exploring it: you can deliver a more complex visualization if you’ve got a little Hans Rosling right there, to direct your audience’s focus and show them what to look at.
We suggest in the paper that some datasets and some stories are better suited for animation than others. There must be a fit between the story you want to tell, the choice of visualization, and the form of data you want to show. This general thought would stick with me, and is a major theme of my recent book with Miriah Meyer.
Bongshin and John have continued doing research on making data compelling and exciting.
We’ve far from exhausted this topic. I hope that we, as a community, continue to do more research into how data storytelling works — to collect examples, learn what it is that makes the experiences so compelling, and to keep digging into what makes an exciting visualization.
I’m delighted to learn that Benjamin Felis was live-cartooning my talk (as well as all the other Test of Time awardees). I’ve never been live-cartooned before!
