Lessons learned: usability testing of the Dutch COVID-19 dashboard
In the face of the global pandemic, communicating reliable data that would allow rapid response to the crisis and decision-making was crucial. But it was also essential to find ways to make data accessible and understandable to everyone, from healthcare professionals to the broad public.
To identify usability issues and evaluate the accessibility of the dashboard, and establish how people find interaction with the graphs, we conducted 2-weekly usability testing sessions in collaboration with an external research agency. We specifically wanted to know how easy the dashboard was to use and interpret. What are some of the most important lessons we learned about data communication from conducting usability tests on the Dutch COVID-19 dashboard?
A graph is only a part of a story
After the outbreak, there was a lot of public debate on the COVID-19 data. People wanted to know the number of positive cases, hospital admissions, but also what insights they could take from that data.
We conducted usability testing with participants from different backgrounds: workers in medical and non-medical fields, people with different educational backgrounds, and also younger and older people. The results lead to an understanding that a number on itself was not enough to understand the situation. People had many questions arising from looking at the numbers: What does this number actually convey? How is it collected? Is it trustworthy? Answers to these questions were necessary to understand the real data in the graph. Contextual information was the first step to understanding and interpreting the data correctly and also seeing how it fits into the bigger picture.
To solve this, we added explanations of the definitions and included articles introducing the broader topic:
Multiple ways to present the same number
During usability testing, common feedback was that displaying data in the format of ‘cases per 100.000’, which is commonly used in science, was challenging to comprehend. While people are more familiar with percentages, these numbers (e.g. infections, hospital admissions) were too low to use this as standard practice. An alternative was to use absolute numbers, yet these didn’t provide the opportunity to compare municipalities, for example. What was the way to solve this? In the beginning, we thought we should stick to one way of showing numbers to prevent clunkiness on the page.
However, in this case, there was no single solution and we took a more flexible approach. On the dashboard, we always showed metrics in multiple ways, to make them accessible to both, scientific and broader audiences.
Jumping straight to numbers
To help understand the data, we added introductions to the graphs. However, these were consistently neglected. Users wanted to read data first and later they did not return to read explanations. We then provided more cues and hints throughout the platform that guided users toward understanding the data:
Summary
Mostly, our journey taught us that the COVID-19 dashboard was more than just a central place to publish numbers on the outbreak. It was essential to add multiple layers to it to make sure it was accessible to a wide audience.
People process numbers in different ways, therefore it helps to show both numbers and percentages, provide introductions, visualize numbers and include images, connect big numbers to the personal context, and add stories that show the overall context of where a number comes from and what it means. Using these methods, we provide needed context in multiple ways and this helps people to interpret data better.
In order to create a truly powerful tool that is data-heavy but also understandable to broad audiences, it’s necessary to observe and listen to how people interpret graphs. This way, dashboards can become fundamental in decision-making and support behavior change.
