Judgement day
A conversation between judges of the Information is Beautiful awards on how they evaluate data visualization submissions
The Information is Beautiful Awards have been running since 2012. In that time, a panel of expert judges has tried to evaluate thousands of data visualization submissions to determine which were the most impressive for the year. I had the following conversation with fellow UK data visualizer Nigel Hawtin about our experiences as past judges of the Information is Beautiful Awards.
We wanted to see if there was anything we could all glean about winning these and other awards that would prove useful not only to people interested in the competition but also to data visualization practitioners more generally in their day-to-day practice.
The conversation took place on 6 March, 2019, in the #critique channel of the Data Visualization Society slack.
Duncan Swain (DS): What was your experience like judging the awards, what did you like about or not like about the judging process?
Nigel Hawtin (NH): First time I had done the judging of IIB Awards, after watching and following for a few years…
NH: My first reaction was wow! how many?
DS: Exactly! When I did the first year’s IIB awards there were around a thousand entries…
DS: So, I guess that tells people that judges are looking at a LOT of entries… so keeping entries, short, concise and to the point is super important
NH: Yes, realised very quickly that it would take some time so I went through the categories quickly and decided that the only way to do it properly and to give the pieces justice would be to set aside a weekend and see how it went.
DS: And even with time set aside you really have to limit the initial exposure to each entry to no more than minutes
NH: Exactly It wasn’t just the initial look of the visual that appealed to me but also the title and quick explanation of the piece, these work so closely together. Sometimes the text explainer made the impression, sometimes the visual and sometimes both.
DS: I spent a loooonnngg time on the first year’s entries (and also subsequently slightly less on the second and third year’s judging but largely because there were less entries…)
NH: Yes, I can see that that happening
DS: 1000 entries x 2 minutes = 33 hours…
DS: which is almost an entire week
NH: yikes
NH: I must admit I spent probably more time on the subject categories closer to my heart…science, environment, tech etc
DS: Some things you can skip through super quickly eg if things are behind a paywall and the details to get in aren’t right, or if the links sent aren’t correct — it’s surprising what can be missing from awards entries, and you don’t have time to search for the right links or ways around a paywall…
NH: Some categories I found easier to get down to my initial top 15…others were really difficult to get down that far…before starting on the top 5–10!
DS: Yes, as soon as you start narrowing down everything slows because you’re obvs selecting the ones that interest and work for you so you’re prepared to spend more time with them
DS: Did you set up criteria to judge by?
NH: Yes sort of, initial look and/or headline was most important to me (did it get my interest), then was it easily understandable or did it have that quick hit (with the time constraints, I didn’t want to spend too long trying to understand) then did it do what the expanation/header said it would do or say…then I may spend more time admiring, clicking, looking, reading or watching
NH: Graphics in another language are great for that reason!
NH: Just want to emphasise that sometimes the title or short explanation made me invest time in the vis — it’s not just the visual bang!
DS: Understood! And I think that was similar for me — though judging foreign language entries a a whole other question, it felt slightly wrong to me that I couldn’t understand what was half of the content (text v visual) and I felt that was kind of unfair (edited)
DS: The words may have been totally wrong, misleading, full of typos…
DS: All of which would affect my scoring if I could spot it
DS: But, to return to the criteria question, mine were really focused on:
- Did I understand it?
- Did the story grab my attention?
- Was there a compelling takeaway from it? Was I learning something I didn’t know before?
- Was the device appropriate to the story?
- Was it aesthetically pleasing (these were the Information is BEAUTIFUL awards after all)?
NH: Similar then. ‘Was the device appropriate to the story?’ assume you mean visualisation type and feel
DS: Yes, that’s right — basically does the visualization of the data work which links to the ‘is it understandable?’ question
DS: One thing I do think awards struggle with (and not just viz awards, I’ve seen it when judging the Webby Awards and the D&AD Awards) is categorization
NH: Agree. I’ve had instances in the past when I really wasn’t sure which category to submit in. Not just subject matter but medium as well.
DS: For example whether you categorize by platform (eg best mobile, or desktop, or printed, or physical) or by type (eg data journalism, interactive, maps etc) then you’re not necessarily judging like with like
DS: Exactly!
NH: Not sure what can be done about this… some awards are trying to change…
DS: Agreed, I think it’s a tough nut to crack
DS: I do love the long lists that IIB publishes every year during the build up to the awards
DS: I always find that super interesting to dig through to get a picture of the state of the industry
DS: It’s also super useful for prospective entrants to see what the entries look like, what’s worked about them etc
DS: I feel there’s a lot to be learnt just from trawling through the backlog of those
NH: Yes, I find myself spending too many hours jumping around the long lists when they’re released. As you say, a great overview of what’s been going on in the past year and an opportunity to see emerging trends and styles as well as discovering new designers and artists.
NH: This goes for many of the awards websites and books. I especially love going back through the Malofiej awards, both online lists and the books.
DS: My background as a journalist means I spend a lot of time reading all the text on any viz i see too
DS: As you mentioned @NH, any headline and subhead are super important
DS: Are they setting up the story in a way that’s both accurate and engaging — does it grab attention and does the piece answer the promise of the headline or title
DS: But it’s also about all the text across the piece
DS: Axis labels, captions, sources and footer notes
DS: All of this is super important
NH: All of it super important…does it read well (edited)
DS: Is it consistent in tone and accurate?
DS: Are there any typos?
DS: Does it all aid in my understanding of the piece as a whole?
DS: Visualization is obvs primarily a visual language but to me that means that any text that is used needs to be super tight and optimised
NH: Big yes to that…, even down to, do the line and word breaks make sense!
NH: Editing of words, data, visuals, colour etc is super important and as important as each other.
DS: Yep — print journalism teaches you to avoid widows at all costs which is nigh on impossible in a responsive world where line lengths vary according to screen size…
NH: 😁
NH: The really tough job was getting my long list down to the top 5 or so in each category. I went over and over many of the entries time and time again trying to find something that would push them up or pull them down the list.
If you’re interested in following up on this conversation feel free to reach out to @nigel.hawtin or duncan swain or on the slack by joining the Data Visualization Society.
