Judging the IIB awards

Every year, the Information is Beautiful awards is a major event in the visualization world. Not just because they recognize some of the most amazing work, but also because they reveal an incredible number of pieces which didn’t necessarily get a lot of attention during the year. And so, each time the long list has been published, I’ve spent hours exploring what I missed.

This year, I have been a judge for the first time and reviewed all the entries in the long list to help form the short list. Here’s what I can tell you about it:

My process

5 criteria

As judges we were asked to pick 5 to 15 entries for each of the 10 categories. I decided to grade all my entries according to 5 criteria:

  • well-crafted: is the entry well executed? Is it balanced, have the right spacing, type, colors? Is it free of bugs and imperfections?
Here’s a very well-crafted entry
  • beautiful: from a purely aesthetic point of view, how good is it?
A very beautiful entry
  • interesting: how interesting is the process behind this entry? This can include the amount of work behind it, the gathering or processing of the data, the approach, etc.
A very interesting entry
  • compelling: how good is the entry at grabbing my attention? If it’s a story, how much do I want to read it, if it’s a tool, how much do I want to use it?
A very compelling entry
  • innovative: How novel is this entry? Does it push the envelope and bring something never seen before, or is it conservative?
A very innovative entry

How about time?

I had one week to review all entries, and used all the free time I had on this so around 25 hours. This means an average 2.5 minutes on each of the 600 entries, and that includes: reading the blurb, navigating to the entry, waiting for the page to load, read and interact, then enter grades. It’s really not a lot but it’s physically impossible to spend more!

How do criteria relate with each other?

This chart shows how each pair of criteria is correlated. For each cell, the scatterplot represent all the entries according to 2 criteria: across, the criterion of the column, down, that of the row. Here’s the code for the curious.

All criteria move together.

More beautiful entries tend to be more interesting, better crafted visualizations are more compelling, more innovative work has higher scores, etc.

10 years ago some people still argued that beautiful visualizations were probably less “effective” than austere, by-the-book ones. That wasn’t true then and it is not true now. Everything else being equal, beautiful visualizations are better.

Innovative and interesting visualizations scored higher.

There were some entries which were beautiful or well-crafted but which I didn’t score very high — That’s the lower-right quadrant of each scatterplot. But there were no very innovative entry or very interesting ones which didn’t get a high score. That’s possibly due to my biases — I’m likely to score high entries that push the envelope.

Beautiful visualizations are well-crafted.

The 2 criteria which are the closest are well-crafted and beautiful. There are is only 1 entry which I found very beautiful (> 8/10) but not well crafted ( < 5/10) — that was because, even though the visual execution was good, the app itself was janky. There are no entry which I found very well crafted and not beautiful.

Often, additional polish (alignments, spacing, choice of colors…) could really make or break an entry. Some visualizations had bold aesthetical choices and as long as they are well-executed this turns out great.

What makes a compelling visualization?

A little bit of everything. Beautiful, well-crafted, interesting and innovative visualizations tend to be more compelling but it’s not enough.

But that’s not enough. Beautiful or innovative entries can be intriguing, or disorienting. Poorly-crafted entries typically prevent the magic from working, but just being well-made is not enough to woo a user either. Likewise, interesting entries (for instance, those with a really good topic or dataset) tend to be more compelling but that in itself is not sufficient.

What works however is work where there is a well-thought narrative or angle; where there is a series of steps I can easily follow to get into the visualization. Data journalism pieces tend to score much higher on this criteria than others, but they are not the only ones which did well.

The state of scrollytelling in 2018

Not too long ago, whether scrollytelling or interactive visualizations was the best way to tell a story with data was still up for debate.

Here’s what Dominikus Baur has to say on the Death on Interactive Infographics, and Robert Kosara on the Scrollytelling Scourge.

The fact is, among the 39 winning entries, 20 used scrollytelling, when only 3 required the user to click around to explore. (The other 16 were something else, including tall websites you have to scroll through to read but where there is no interactivity triggered by scrolling).

Logically, one could still make a case for creating interactive visualizations to tell stories, but that would ignore the fact that the world have moved past that debate now. The more scrollytelling visualizations there are, the more people expect them, the least likely they are to interact.

Creating data stories in a world where storytelling is now the undisputed dominant form has some implications. Authors just can no longer expect users to explore every nook and cranny of their creations. That also means that authors have to control the experience more as opposed to let users figure it out, plan the narrative and only present the required data at each step (as opposed, again, to flood the users under a data deluge and let them navigate).

In the age of scrollytelling, users are not deprived of control over how the story is told, rather that control is simplified and streamlined. There are other forms of visualization which integrate nicely with this form of communication:

  • animated GIFs that can summarize a story by cycling through the charts that a reader would see when scrollytelling,
  • Self-playing short videos, integrated in the flow of the story, that provide breathing space for the reader,
  • Transitions — when a chart or map doesn’t move on screen, but instead changes from state A to state B as the user scrolls.

Yet: making scrollytelling stories is hard. As far as I know, there is no off the shelf tool that can support scrollytelling with data (which could be an opportunity for, say, Tableau or Flourish). There’s no great coding solution either from the examples where I read the source code. The pudding team put together this article last year and the situation hasn’t changed even though creating visualization on the web, itself, tends to be more ambitious complicated than one or two years ago.

A super sophisticated, smooth, reliable implementation is not necessary to get the “scrollytelling effect”. I’d like to give a shout out to this entry by Christine Quan:

Airweets: an Emoji Story

that uses a very simple form and still is able to deliver an organized sequence of ideas, one at a time.

Conversely, using scrollytelling isn’t enough to make any story good. Mike Bostock’s recommendations (from 2014!) are still valid, though I haven’t seen in the IIB entries of this year a single example that blatantly goes against them.

Finally, the opposition between interactive visualization and scrollytelling could become more blurry as more and more entries appear to have been designed mobile-first:

20 years, 20 visualizations from the South China Morning Post team

In the collection above, all stories are “bit-sized” and are meant to be consumed in a linear, straight-forward way. Sometimes this is achieved by scrolling, sometimes by tapping and this choice just becomes a detail rather than a defining strategy.

Some things that didn’t work so well

A caveat for this last section — like for the rest, this is only my opinion. For context:

  • Out of all the entries that were shortlisted, 70% were on my shortlist;
  • I voted for 2 out of 3 winners.

For the record, my favorite entry — the world cup predictions from 538 — was shortlisted, but didn’t win.

So I would say I’m roughly aligned with what makes a great entry but not exactly.

With that out of the way, here’s what hurt some of the 2018 entries scores:

  • If it’s behind a paywall, or if I have to log in to social media to see it. I’m willing to make reasonable efforts to judge the entry but I need to be able to access it!
  • The flip side of this: some entries were physical installations or objects or books, and I just don’t have good material to judge them (and again: very little time!)
  • I’ve spent a lot of time with some of the entries. But if I have to spend a lot of time to get it… it’s not going to work for me.
  • In general, I didn’t score well the entries that threw a lot of data at me. Many entries used small multiples or glyphs (or small multiples of glyphs) as a technique to encode large and complex datasets in a limited space. And this sometimes work! For instance I liked A Night Under the Stars by Jordan Vincent. But using either small multiples or glyphs is not enough to rein in complexity. As a reader I need to be given an angle with which to approach the visualization. The Freedom of Expression entry by Nadieh Bremer is interesting in that regard because glyphs are unambiguously sorted from top to bottom so I can scan the ensemble easily.

So see you next year?

Before I started judging I thought I was getting the most of the awards by spending an hour or so on the site when the winners were announced. But it’s been extremely rewarding to go through the long list and see all the amazing ideas even in less ambitious projects. So for everyone interested in visualization, and interested in data narratives in particular, I really encourage you to jump in as soon as the long list is announced and explore! I’m really impatient to see next year’s entries.