Are data visualisations for scientists or policymakers?
Our scientists are publishing sophisticated data visualisations for policymakers to use. But are policymakers using them?
Note: In Part 1 of this post I interview Annie White and Nil Tuzcu of Harvard’s Growth Lab about their use of data visualisations for policymaking. In Part 2, I set out the context (what I was looking for, and why), and then summarise the resulting guidance we’re giving Knowledge4Policy publishers.
Part 1: Interviewing Annie White and Nil Tuzcu, Harvard Growth Lab
Last November I attended the EUDataViz conference on the use of data visualisations by the public sector.
There were many excellent sessions, including this one presented by Annie White, Senior Product Manager of the Growth Lab at Harvard’s Center for International Development.
It struck a chord (see Part 2, below) so I reached out. Annie and her colleague, data visualisation designer Nil Tuzcu, kindly agreed to answer my questions by video-conference a few weeks later.
(ML) Briefly, what is Harvard’s Growth Lab?
(AW, NT) The Lab is a Harvard University research centre studying the dynamics of economic growth and translating those insights for policymakers in developing countries and urban centres. We place increasing economic diversity and complexity at the centre of the development story, and investigate how countries can move into industries that increase their productivity.
We [Annie and Nil] are part of the Lab’s Digital Development & Design Team, and translate the work of the research teams into online tools. The team’s work has been featured in the Washington Post, Bloomberg, the Financial Times and Harvard Gazette, and was short-listed for an Information is Beautiful Award in 2018 and 2019.
Tell us a bit more about the Atlas you presented at EUDataViz
The Atlas of Economic Complexity is our flagship digital product — it reflects our conviction that while academics publishing research papers is essential, we will have a greater impact if we provide tools which both make the data available and ‘embody’ our research, in forms non-experts can use.
we will have a much greater impact if we provide tools which both make the data available and ‘embody’ our research in forms non-academics can use
The Atlas is a powerful data visualization tool, used by policymakers, economists, academics, analysts and journalists to explore global trade flows across markets, track these dynamics over time and discover new growth opportunities for each country. It also plays an important role in capacity-building, as a support tool in both Harvard’s economics courses and the Executive Education courses attended by policymakers, economists and business people.
It contains trade data for 250 countries and territories, classified into 20 categories of goods and 5 categories of services, from sources like the UN and IMF. Combined, this results in coverage of over 6000 products worldwide.
We saw our 1 millionth user last year, and have users in almost every country on the globe.
At EUDataViz you showed two ways users can engage with the Atlas: “Explore and Explain”
The Atlas offers users different ways to engage with the data and concepts, depending on their needs and expertise. The underlying idea is to present both author-driven and user-driven interfaces, based on a 2010 paper [Segel, E., & Heer, J. Narrative Visualization: Telling Stories with Data (pdf)].
So on the one hand we developed Atlas Country Profiles to tell the story of a country’s economic complexity. They take a user through a traditional narrative, step by step.
These Country Profiles are what we call author-driven: our researchers took what the Growth Lab understands about the growth process and developed a story, or narrative, summarising that process. We depict each segment of that narrative with an interactive visualisation, and combine them in a form that’s accessible for our user base.
Each country profile contains around 100 pieces of computer-generated language
But writing narratives consistently for over 100 countries, and keeping them up to date, just isn’t feasible. Instead, we use computer-generated narratives: a single story template which contains many different possible scenarios, depending on the country and its data. Each country profile actually contains around 100 pieces of computer-generated language.
The insights derived from Country Profiles are aimed primarily at analysts and policymakers, so it’s no accident they require less familiarity with both tool and content. But we also wanted to ensure the content was accessible to students, journalists and others coming to this information for the first time.
On the other hand, users can also access the Atlas’ Explore page. This is more of a “choose your own adventure” approach — the user can explore 250 countries, 6000 products and services across 6 unique data visualisations, so you can imagine the possible permutations available.
This Explore section of the Atlas is user-driven — users can conduct deep analyses and craft customised experiences, aided in part by many more visualisation controls than Country Profiles provide. However, pulling insights from the Explore section often requires a user equipped with a more advanced understanding of the research methodology underpinning the Atlas and indeed the Growth Lab, and so is aimed at a more advanced user.
What traps do you look out for when creating these visualisations?
We’re always asking ourselves “is this the right visualisation?”, because data visualisations can be easily misinterpreted by users, or can even mislead them.
It’s very easy to create visualisation tools overburdened with information and complexity. Economics is complex enough — noone needs visualisations adding to it
Sometimes this is simply a matter of information design. It’s very easy to create visualisation tools overburdened with information and complexity. Economics is complex enough — noone needs visualisations adding to it.
We do a lot of design testing, and can devote entire meetings on issues such as how we label a visualisation’s axes. We have also implemented quite a nice contextual help system, and are currently rolling out a ‘How to Read this Graphic’ tool across more of our visualisations.
visualisations seem to have an aura of credibility — but if the underlying data is not good, that credibility is a facade
But it’s also possible that a visualisation that does work for, say, 130 countries does not work for 50 other countries, simply because of data quality. There are some countries where we simply don’t provide certain visualisations because the data is insufficient, and the result could mislead users. You have to watch out for this because visualisations, by their nature, seem to have an aura of credibility surrounding them — but if the underlying data is not good, that credibility is a facade.
You also provide decision-mapping trees for selected visualisations.
That’s for the ‘Strategic Approach’ visualisations, which offer a recommended policy course for each country, derived from its complexity and connectedness to new opportunities.
We help users understand that recommendation using a decision tree, which interrogates the data and takes the user down a series of branches to arrive at the recommendation.
We visualise the decision tree so that our methodology is transparent, and we avoid issuing recommendations without explaining how or why.
Many thanks. Where can readers get more information?
There’s the Atlas itself, which is free and available for anyone to use. You can also check out our About page and our short video tutorial on Country Profiles. On Twitter we are @anniewhite and @niltuzcu_, while Professor Ricardo Hausmann, Director of the Growth Lab, is @ricardo_hausman.
Part 2: Context and Conclusions
Annie’s presentation struck a chord because we’ve been arguing about data visualisations on Knowledge4Policy (K4P) since the outset.
Context: mixed messages
Some EU Commission Knowledge Services had invested in sophisticated visualisations well before K4P was even conceived. While migrating these bespoke applications to the Commission’s web platform proved impossible, we did develop tools allowing Knowledge Services to publish visualisations created using Tableau, HighCharts and Qlik.
Yet all the time, we were wondering whether it was worth the effort: our audience research, carried out as we prepared our Spring 2018 Beta launch, was contradictory.
we were wondering whether it was worth the effort
On the one hand, some of the scientists we interviewed told us that policymakers loved their data visualisations.
But not all: some Knowledge Services told us that that policymakers found their visualisations too difficult to use. On the other hand, the online survey part of our audience research was more positive: those self-identifying as ‘policymakers’ rated visualisations the best possible use of our limited resources.
This was flatly contradicted by the face to face ethnographic interviews and focus groups which followed. By then we had split the ‘policymakers’ audience into ‘policy advisors’ and ‘policymakers’ in our audience research. While the former were reasonably keen on using visualisations to prepare briefings for the latter, neither they nor the actual policymakers we spoke to recommended showing data visualisations to policymakers. As the audience research concluded:
“while those at advisor level might engage with interactive data visualisations, in general access to concise infographic style information was thought to be most useful for senior policymakers”
- Knowledge4Policy: Audience Research (page 50)
This chimed with my personal experience. I had no difficulty imagining policymakers being instantly impressed with the data visualisations the EC’s scientists were developing, while knowing (perhaps only in their hearts) that they’d never find the time to use them. But that was only a personal opinion.
On the other hand, the scientists our Knowledge Services work with do use data visualisations, although many also like downloading data and analysing it themselves.
Hence the question: if the visualisations we’re building for one audience are actually being used by another, is anyone served well?
So what does all this mean for Knowledge4Policy?
The “Explore v. Explain” paradigm slots nicely into the (overly) simple ‘linked knowledge pyramid’ underlying Knowledge4Policy, with “Explain the data” visualisations prepared for policymakers ‘linking down’ to “Explore the data” visualisations for scientists using the platform.
This is easier to say than do using off-the-shelf tools, rather than bespoke developments like Harvard’s Atlas — each visualisation takes time to create, so making two different visualisations for two different audiences might take almost double the time.
One cost-effective solution, reflecting our original audience research, could be to simply present static infographics at the top of the pyramid, linking down to interactive visualisation for users wishing to explore the data themselves. We’ll support this in the new content types we’re currently developing to reinforce K4P’s ‘policylayer’.
The ongoing challenge will be avoiding data misinterpretations
The ongoing challenge, as data visualisations are increasingly used in policymaking, will be to avoid data misinterpretations. This means reinforcing Knowledge4Policy with:
- content, providing interpretations of the data alongside each visualisation, which is where a “one message per visualisation” rule needs to apply;
- online help, empowering users to use visualisations correctly, backed up by expert and community support (when we roll out community features);
- information design: constantly asking ourselves “is this the right visualisation?”, and incorporating audience testing as a matter of principle
- data quality assurance, to avoid presenting visualisations which appear more credible than the underlying data warrants.
Multidisciplinary capacity-building will clearly be essential.
What do you think?
We’d love your thoughts, ideas and useful links, so don’t hesitate to share them by Responding to this post, submitting your own or by Tweeting to @EUScienceHub.
