What People Really Mean When They Talk About Data Overload
“Confusion and clutter are failures of design, not attributes of information”
– Edward Tufte
People throw around the term data overload all the time to describe a problem in a product. On the surface, this seems pretty direct. Users have trouble sorting through the data presented to them and make bad decisions routinely. They look at the UI, see how much data they were faced with, and immediately declare that there was too much data to make the correct choice.
This leads to pretty consistent design responses: Provide a control that lets the user filter the data (or filter for them it so the data is unavailable altogether), paginate, or implement a search. Perhaps the system designer introduces an analytic to make sure the user sees the important data first (with the implicit assumption that the product designer will be able to figure this out for all situations). The designer tests the system and gets positive reviews and improved decision-making. The user sees a simpler UI with a manageable amount of data and declares that the system is much better.
This victory, while nice, is a bit hollow. The improvements are generally small and don’t address the root cause of what the user is experiencing. The problem is not that there was too much data, but, as in the quote from Tufte states, a problem with the design, and finding ways to remove data is almost definitely the worst of good alternatives.
Human capabilities are underrated
Humans, despite the traditional understanding, are amazing information processing beings. We have a tremendous capability to take in large quantities of input from a variety of sources and make good decisions about where to focus our attention and what actions to take. If not, we wouldn’t be able to function in our day-to-day lives. Just think about what life is like walking down sidewalk on a busy street. You have storefronts with signs trying to catch your attention. Traffic lights are signaling cars and you see the walk / don’t walk signs for pedestrians. Other people are walking all around you. You hear myriad noises from all directions. In short, your senses are being bombarded.
Despite the high sensory input, in these situations you rarely fail to get to your destination (if you have one). Why? Because you have delicately evolved mechanisms to process information, filter out the irrelevant, and focus on what’s important. You ignore the faces in the crowd you don’t recognize and the traffic signal. You don’t care about the walk sign until you get to the intersection. You don’t really notice the gentle slope in the sidewalk, even though your body has adjusted to start walking uphill. You focus on the task at hand — getting to that new restaurant you’ve been meaning to check out.
Good design utilizes these built-in mechanisms to help the user accomplish their goals successfully. They use the human’s innate capability to filter irrelevant information rather than try to guess and filter it for them.
What ‘Data Overload’ really is
When people talk about Data Overload, usually one of three problems is present. The first problem is that the system presents irrelevant data to the user. At least some, and possibly all, of the data provide little decision-making value. This is a result of a product designer who has little idea what data is relevant, so they collect whatever data seems to make sense — or more likely, whatever data is easy to collect that is tangentially related to the problem. This seems like data overload because there is so much noise to wade through that the signal is impossible to find.
The second problem occurs when the product designer doesn’t know how to present the data to the user. Data visualization is a science. There is a large body of research about the human perceptual system and how this can be leveraged in helping users understand their data. Not all charts can be used for all data sets. People are adept at identifying patterns in the world and data visualizations can be used to support this pattern recognition. But when the wrong visualizations are used (or no visualizations at all), the data can seem to be overwhelming.
The final problem when people talk about data overload is that sometimes there isn’t enough data. This seems counter-intuitive, but data is best understood in context. Take for example an application that helps you manage tasks. Many of these applications provide ‘Due Date’. This is an easy piece of data to collect and makes sense to show in the product. However, by only showing due date, the product makes the user do a lot of the work. What users really need is time/days until the task is due. This means that the product needs to capture current date, find the difference between current date and due date, and then show this new value to the user. Sure, the system needs to gather and store more data, but this makes the user’s decision-making easier.
These problems are even more evident when they are found in combination. It’s really easy to use the wrong presentation approach when you don’t provide enough data. And when you don’t know what the right data to show your users, it’s really easy to give them the wrong data instead.
Helping users manage their data
How do you get things right? Rather than start with the data you have and figure out how to show it to the user, start with the decisions the user is making and figure out what data they need to make that decision and what context it should be in. This gets tricky, since sometimes the ideal data is not available or obtainable. Then you need to compromise, while maintaining a record of what you would like to get in case it becomes available. You should probably even inform your user that they are getting an approximation of what they really need, to help them understand the gaps in their knowledge.
Once the information is identified, the key is to then figure out the presentation device that best reveals to the user the relevant data patterns. Don’t pick a chart for your data that you think looks good, pick the right chart for the decision and make it as evocative as possible without reducing the decision-making capability of the user. User decision-making comes first; styling comes second.
I hate hearing the term data overload, because it assumes humans are feeble and that designers have no responsibility. Instead, as designers we have the duty to understand how humans think, and to understand the decision-making needs of the user in the domain we are working in. We need to understand how human perceptual mechanisms work and how that impacts how we encode the data that users need. We need to take advantage of the power of our users and design in a way that truly supports them.
