Charting new territory for data journalism

Data Explorer is a tool for visualising the vast datasets from the ABS, UNESCO, UKDS and OECD. We aim to make public data as accessible as possible.

Background

A year ago, I was helping my wife with a common problem in her job as a journalist — wrangling data. More specifically, she was trying to build a chart using Excel and Australian Bureau of Statistics (ABS) data from a poorly formatted PDF.

As a developer, I saw her process and felt an uncontrollable urge to streamline her workflow. After doing some research, I discovered the ABS had a public API (Application Programming Interface, a software intermediary that allows two applications to talk to each other). In a weekend, I was able to build a rough prototype website that could create instantly charts from over 300 datasets.

My wife was impressed and suggested I submit this project to the The Walkley Media Incubator and Innovation Fund. She was already working on a couple of submissions with her colleagues at The Conversation. She may have regretted telling me about the competition — I ended up getting one of five grants through the Walkley Foundation.

With the funding, I’ve spent the last year turning my rough prototype into a production ready website. I’m far from done, but I’m happy to announce https://dataexplorer.io is live.

The data

In my original pitch, I only used ABS data via an obscure API format called SDMX (Statistical Data and Meta eXchange).

To my surprise, I found out that OECD, UNESCO and UKDS also use the same format. Not so obscure after all. In an instant, the project went from 300 datasets to over 2000.

This influx of data presented some new user experience (UX) challenges, but the core application remained mostly the same.

Search and create

Data Explorer has a simple search box to find datasets by organisation. Not all datasets are suitable, so I have selected the best ones and labelled them as featured.

Every dataset has several dimensions to adjust the data. These dimensions are contextual, so they change depending on the dataset you are looking at.

For example, the ABS Labour Force dataset has dimensions such as region, data item, sex, age and adjustment type.

Meaningful comparisons can be made by selecting multiple values within a dimension. To illustrate, the chart below compares male and female data unemployment rates.

This one takes it further and compares several regions.

Every combination of every dimension can be accessed via a unique URL. Bookmark or share the link to go back to the exact chart.

The above charts can be accessed below:

• Labour Force — Unemployment Rate (%), 15 and Over, Trend, Females
• Labour Force — Unemployment Rate (%), 15 and Over, Trend, Males and Females
• Labour Force — Employed Full Time (‘000), Persons, 15 and Over, Trend, Multiple Regions

Data visualisation is notoriously difficult on mobile devices. I paid particular attention to this issue, taking inspiration from Google Trends.

Mobile view of the ABS Labour Force dataset

The charts themselves are either line or pie charts. There will be more types of charts to come. They update fluidly, responding to changes in dimensions. The goal here was to allow users to quickly fine tune and encourage exploration of the data.

Once the user has selected a chart, they can either download a CSV file of the data or an image of the chart.

Sharing the chart on Twitter will display the actual chart in the card image. I’m hoping that people will use this feature to create short data stories like the one below:

Feedback

I’ve only had a small group of beta testers try Data Explorer so far. There is still a lot to do, but I’m ready to release the project into the wild and get more feedback from a wider community.

@dataexplorerio will be the main contact point on Twitter. Get in touch if you have any suggestions, feature requests or noticed any bugs. I’ll use this platform to help prioritise new features.

For the developers out there, this project is built entirely on open source software such as React and GraphQL. Most of the project itself is available as open source on https://github.com/unkleho/data-explorer. I’ll follow this article up with a more technical rundown in the near future.

Credits

Many thanks to the Walkley Foundation, Isentia, Google News Lab, my early beta testers, Aidan Temple for coding help, all the organisations for making their data open and all the open source software maintainers out there. Most important of all, thanks to my wife for providing the original inspiration and ongoing support.