How John Burn-Murdoch’s Influential Dataviz Helped The World Understand Coronavirus
An interview with the Financial Times data-journalist about his experience visualizing the COVID-19 pandemic
One hears the word ‘unprecedented’ a lot these days. It’s as if the language we use to explain our world is breaking down and superlatives just aren’t able to keep up with the new reality brought to us by the coronavirus pandemic. Living through the past month has brought an avalanche of hard to answer questions, as we’re limited to data that is sparse and difficult to analyze.
Many have noted the importance of data visualization in helping people attempt to make sense of it all with a few data journalists contributing significant impact. One of them is John Burn-Murdoch of the Financial Times (FT), whose breakout moment came on March 11 when his first log scale chart comparing the trajectory of infection rates between countries helped millions of people around the world understand that the pandemic was a trend just beginning in England and the USA.
John continues to analyze and report on the coronavirus pandemic every day, so Nightingale is incredibly thankful that he took a few moments out of his busy day to speak about his experiences.
Jason Forrest: How did your coverage of the coronavirus begin?
John Burn-Murdoch: This is the biggest story as a data journalist that I’ve ever encountered, this is just a story that when this comes into the news you just know, this is our story. There’s no process where we’re assigned this, it’s just something that happens and as a journalist, you say “Right, I’m on it.”
JF: How did you determine that these would be the charts you would use to tell this story?
JBM: It’s always difficult with cases like this to try and retrace one’s steps and work out exactly where different bits of ideas came from. Because what happens with stuff like this, especially in data visualization (and especially for a massive story like this) is that everything comes from somewhere.
There were multiple conversations going on when I made the first iteration of this chart that I’ve now made about 50 times. One of them was a conversation with one of our reporters who was interested in comparing the Spanish and UK daily numbers of cases and deaths in relation to Italy. I think it was on the 10th of March if I remember correctly. My response to answer that question was to see if I can get the whole table's worth of data to her to show what we’re looking at.
In that initial email, I made a couple of versions of the charts that we are now doing every day. One was on a linear axis and one on the log axis and these were both just my way of saying, ‘here’s the data on about five countries and here’s what it shows in terms of the inevitability,’ that all countries were heading down the same road as Italy.
So the first version (0.0) was just a very rough ggplot R graphic. Then I think it was the day after that there was a conversation in our morning news conference with all the editors where someone said, “Should we have the FT have a definitive chart which says where everyone is in relation to Italy that tries to answer that question of whether we’re all heading in that direction.” As soon as I heard that sort of appetite and interest from the editors, I thought “oh great, this is where I take the chart I made yesterday and do a more finessed version of it.” That was how it started coming to shape within the FT.
But I’m fairly sure that I’d seen all of the constituent parts of the chart floating around in other people’s work. I know that someone called John Minton who’d been producing a lot of graphics already by this point, which used the idea of a starting point of 100 confirmed cases. I’ve encountered that in a lot of conversations with epidemiologists, that on a chart like this you can’t really start at the first case because you can’t say that a country has a fully-fledged outbreak when it only has one case. So if you want to make comparisons between countries, you’d want the chart anchored to an epidemiologically similar starting point of 100 cases.
Then as for the log axis — as I said, the original chart that I did was in one log and one linear — it was immediately clear from doing it that the log one was going to be the more useful. Otherwise, you had loads of countries that just got completely lost and squashed into the bottom left-hand corner. In addition, that I’ve explained on Twitter in particular, when you’re dealing with a virus, a virus spreads exponentially not linearly, so this is just a rational path to take.
The only other point on a log scale is that, for me, it’s about the amount of visual bandwidth that you have to deal with when making a chart. When you use a linear scale to plot exponential growth a lot of that visual bandwidth is taken up by that increasingly sloping curve. If we know that all of these curves are going to be exponential curves then using a linear scale with the linear y-axis would use a lot of the visual space just to show that all of these countries are seeing cases at an exponential rate. That feels like a waste of that space since we know that’s the case anyway. By using the log scale you use your visual bandwidth to see the slope of the different rates of growth. You’re not wasting so much space looking at all countries just making the same curve.
JF: It’s a very easy chart to “see” — to come up with a “so what” from. How was the initial reaction to it when you started to share it amongst the FT staff and then when it was published. Was it an immediate success?
JBM: That’s the thing, this one more so than anything else I’ve done, this one has set the direction of my last month’s work.
The first version that went out, it was the cumulative number of cases over time. That one was making the point that pretty much all the western countries look to be on the same trajectory as Italy. Of course, at that time, Italy was seen as this Ground Zero of the whole thing. So this chart was making a very emotionally powerful point that this country that we all agreed was going through something pretty terrible was just a few days down the same path that all of our countries were also setting out on. I think it was that message as much as the choice of geometry which really seem to cut through.
That first version got this huge engagement, a huge outpouring of responses and everyone all seemed to be endorsing the message in the chart itself. So from then on it just felt like the point was to drive the inevitability of coronavirus and how countries are going down the same path as Italy, it would be natural for us to update this from one day to the next.
We already had, at the FT, a coronavirus tracker page. I think it was just a map and a data table, but we already had something which was a daily updating page showing the key figures on coronavirus. It was easy to just say ‘oh, well, we now have a new chart out there which seems to be resonating very strongly, so we’ll simply add that to the page here’. The rest is history since then it’s just been a case of making daily updates to this chart adding additional sister charts and iterating on the existing designs.
JF: You said that you’ve made basically 50 changes to the chart over the last month. How has your relationship with the actual chart or the work changed in that time?
JBM: I love the wording of that question. I think the idea of me having a relationship with the charts and the work is a very apt term to use. Because especially during this period of lockdown, where we will work from home, it does feel like a very intimate part of my daily routine.
But yeah, it’s changed a lot because at the outset this was a case of an urgent need to keep updating this graphic which people felt strongly about, and to keep emphasizing the point of countries following Italy’s outbreak trajectory. But there are many ways in which the dynamic of the relationship has changed. So part of it is that the story itself has changed. You’ll see that we now lead with these slightly different charts, which are looking at daily numbers of infections and deaths, rather than the cumulative totals because the story really feels like it’s moving on. Instead of saying which country has the most cases or deaths, or how many days behind country X is country Y — it’s now a question of when does each country reach its peak in terms of infections or deaths and this question of when might it be possible for restrictions to ease.
But the way that these charts are perceived has obviously changed a lot as well. For me, this is something completely unique, in that we now have a piece of visual journalism, which people far beyond the immediate readership of the FT have come to see as a part of the way that they experience news coverage of coronavirus.
There was The New York Times needle in the 2016 election, of course, and they have brought that back at times for other elections so it’s become synonymous with election graphics and you know, it feels like there’s something similar happening here. Where a few nights ago, I only published a subset of the charts I usually do, because it already got extremely late in the night when I was getting the stuff out there and I was inundated with people saying “Where are the other charts?” We get hundreds of emails and messages every day now exclusively about these charts — that’s emails directly to me, that’s emails to our team address, and a lot of my Twitter direct messages as well.
So I think the biggest change for me is to be presiding over something which is now genuinely part of the lives of thousands and thousands of people as well as the typical demands according to the editors at the FT, there’s now this additional secondary set of stakeholders. And we feel — even if only on a personal level more than a professional one — that there are people here now who have come to expect something, who have very interesting and valuable feedback in comments multiple times a day or other things we could be doing with these charts. As the person who sticks pixel to paper, as it were, that’s just a very different dynamic to anything I’ve had before. In the past, I’ve made plenty of charts that have been about important emotional subject matter, but they tend to be a case of ‘publish and forget’ — you put something out there and it’s done. Whereas, here we might want to make small changes once a week or even once a day. It’s a much more involved process.
Just as an aside, I think once things settle down, this is going to end up being an amazing resource in terms of public engagement with and response to data visualization. We’re now sitting on well over a thousand bits of written feedback to our chart and people saying what they like and what they don’t like, and have we considered x and have we considered Y.
So yeah, on day one I thought I was just making a standalone chart that made an important point and on day 30 — whatever we’re on — this is now a product with a huge audience of people both at the FT and in the wider public who have come to rely on and to have expectations of. Just as an aside, the page on FT.com that these charts sit in is by far the most read page of the FT website ever. I think we’ve certainly got enough ‘credit’ built up here that anytime anyone in the future questions ‘what is the value of data visualization in the newsroom’, we’ve got several million answers here.
JF: That’s amazing, totally amazing. So you said that you’re getting just an overwhelming amount of input and response from the broader global audience. What kind of toll is that taken on your personal life?
JBM: It’s an excellent question. The nice thing about this has been that it has really been a brilliant incentive to streamline loads of workflows. I’ll answer this in several ways, but that has been quite nice.
The first versions of this chart were made in ggplot and then a load of tidying it up in illustrator. I ported it over quite quickly into D3, because that allowed us to make some different styles and sizes of the graphics quickly, but a big breakthrough was a couple of weeks ago when I managed to clear the last couple of hurdles, and now the entire thing is done in D3 in the browser without me needing to do any fine-tuning and so in terms of web saving 15 minutes here and 20 minutes there, it’s been really nice to keep doing that fine-tuning and that streamlining.
It was a necessity to do that because it was otherwise a huge load on me in terms of the amount of time I have to stand this up this every day. One of the particular challenges is that when countries publish their daily updates in the data, essentially everything comes in the evening time over here. Today, for example, and if I take a look just now, we’re still working on data from France, Germany, and Turkey in terms of our major countries of interest. Of course, in the US we only have partial data for so far. For me to update the charts so that they are timely — so that the numbers in these charts match the numbers in our news stories — I really need to be updating in the evening. So typically I start work updating the charts at 6 p.m.
This just illustrates how much of a nerd I am, but last night I timed every sub-task of that chart updating process so I can try to see where I can make efficient saving next, so this is just after a conversation with my other half, and we were you saying how I can get back some of my evenings.
Starting about 6 p.m., it ends up being about 1 hour 45 minutes, which is about 1/3 obtaining and cleaning up the related data, 1/3 making sure the charts are rendering as they should and moving some annotations around. Then 1/3 putting those into our new story and writing any commentary played alongside it. It’s been a big thing to deal with because the fact that I do that from 6 to 8 p.m. doesn’t change the fact that I still have my main day job.
We have our main team meeting at 10 a.m. and the rest of the day is doing all of our huge amounts of other daily coronavirus reporting. These charts and this page is just one tiny bit of what our team is involved in at the FT. At the moment, I’m involved in other stories looking at the impact of different countries lockdown; at the impact of coronavirus on the environment and on pollution; and all sorts of other bits and pieces and like tracking the gradual reopening of China. Then around 6:00 p.m. when our team slack channel is a stream of hand waving emojis 🖐, that’s the time when our second shift begins for myself and a couple of others.
It’s been a really big deal in that sense. The way I described it to someone else was it’s a bit like doing an election night shift every day for months. So you’ve got some excitement and adrenaline and intensity of covering a fast-moving data-rich story, but you can’t then have a ‘lie-in’ the next day because you’re going to do it again. So yeah, it’s been very intense.
I’ve loved to be as involved in this very intense dataviz story as I have been, but there are plenty of other things that, in both work and in my own life, I’d love to be doing more of that had to be put on the back burner for the last month. We’re still very much going to be covering this story as a dataviz team, but the idea is to make it more of an automated and routine process and reduce the need for this sort of curated stuff that I’ve been heavily involved in over the month to date.
JF: Yeah, that makes total sense. It’s a natural progression into something that’s just more sustainable. Right? I would presume every bit of pipeline you have has got to be well automated and documented. So you’ve had the time to get the technology right…
JBM: …on that last point, that’s something we’re really ramping-up on at the moment. We now have a little box on that page on the FT website where we track when we’ve made changes to the charts and explain them. The idea has been to turn this more explicitly into a product and to make this something that readers see explanations as to why we’ve made changes, and how we’re doing things, and what’s changed where. The evolution of this from a chart into a product — that’s been part of what has made this so unique about this.
JF: Last question: do you have any anecdotes you can share about interacting with so many people about this?
JBM: I’ve exchanged hundreds of messages on this in the last couple of weeks, and there have been some good ones. But one that is less funny and more an interesting example of the sensitivities of the strength of opinion around this stuff: one of our sort of flagship charts in this series shows daily new numbers of deaths attributed to coronavirus and the headline states “Every day brings more deaths than the last in the UK and the US”. Every time you’re making a statement about a country that may be portrayed in a negative light (which some people might take that to be in this case) you’re going to get people who have a bit of a “fan” reaction to that and say, ‘you put down my country’.
The reason this is a particularly complicated issue, in this case, is that the data we’re dealing with here that we get from countries with coronavirus is extremely patchy in terms of its quality and noise from day-to-day. That’s true to such an extent that I personally don’t think that the daily numbers we see and hear on the news every day are actually worth the paper they’re written on. Because from what we know now — even in countries like the UK and the US — the daily numbers and the fluctuations in those numbers of deaths that we get have as much to do with the idiosyncrasies of how deaths are reported as they are to the actual spread of the virus.
For example, every Sunday and Monday in the UK (and I believe this is true of the US as well) the number of deaths reported falls. So every Sunday or Monday — this is a true behind-the-scenes story — I’ll get some readers emailing in and saying your headline says that deaths are rising day by day in my country but on Sunday/Monday the numbers are going down, so the headline’s wrong. And I’ll respond to them by saying, “look, it’s complicated. We believe that the nature of the daily data here implies a false level of precision, and therefore we’re using a seven-day moving average on our charts, which is the better reflection of the sort of week-to-week way that these viruses spread. You’re never going to have a peak day, it’s really a peak period. Our headline reflects that seven-day moving average is still trending upwards”. I’ll still get a reply then from those people saying “I understand what you’re saying, but the fact is your headline is still objectively incorrect.” Then like clockwork, every Tuesday, the backlog of reporting of deaths that have built up over the weekend is released and you get a huge spike in the numbers.
It’s just one of those weird things where the decisions we make in these charts— things like using the seven-day moving average — which is all to try and be more honest and to try to portray a more truthful and meaningful picture of what’s actually happening out there. But people will really focus on those tiny details, even when we know and have explained that those details are actually highly misleading and not really reflective of what’s happening with the virus. But those are the details that people focus on and write a letter to the editor objecting in the strongest terms to what we’ve done — even though what we’ve done is explicitly going towards presenting a more honest picture of what’s happening.
The point here is that this is an issue that people are really spending huge amounts of time poring over and people feel very strongly about. So we get into huge lengthy debates now with hundreds of people about what type of rolling average we should be using, or whether we can truly claim that something was going up every day when it shows the trending up every week and what type of log scale we should be using and that kind of thing. So yeah, the overall point is that the strength of feeling of it is huge and that’s and it’s incredible to be at the heart of that. But it can be pretty stressful at times.
Here is the chart for April 13, 2020:
Here’s a video of John further explaining his various methods and charts:
You can read more of Nightingale’s coverage of the coronavirus pandemic here.
