Let’s Talk About Sets
How data is undermining our ability to design for everyone
Design has an empathy problem.
Whether we realise it or not, many products and services we use every day are built with a particular user in mind: white, male, able-bodied, neurotypical or heterosexual.
This fails to recognise the majority of users and has impacts ranging from minor inconveniences — like soap dispensers that fail to activate for those with darker skin tones — to life-threatening situations — like the misdiagnosis of heart attacks in women.
More often than not, this occurs because the information we gather — such as from datasets or focus groups — to decide how a product should be built isn’t representative of real users.
By not designing for everyone, we miss crucial opportunities to not only save lives, but also improve the user experience for all, and expand our audience by up to four times. We need to acknowledge our failings. And fix them.
Rubbish in, rubbish out
Flawed input data will result in a flawed output.
If you build a facial recognition tool using machine learning but train the model only on images of white people, the model will learn to categorise faces based on white skin tones and features. Present the app with images of black people, and it may struggle to distinguish between them. Take Joy Buolamwini, for example, whose face was not recognised by a facial detection software until she put on a white mask. She now campaigns for racial justice across AI and founded the Algorithmic Justice League.
This movement should be applied holistically across all types of services, as it raises a lot of questions about how design — however well-intentioned — can embody societal prejudices and amplify discrimination.
So how do we spot the flaws in our own datasets?
Don’t just look where the light shines
As put by Kate Murphy in You’re Not Listening: “…the difficulty with looking for answers in datasets is you become like a drunk looking for his keys under the lamppost. Ask the drunk why he’s looking for his keys under the lamppost, and the drunk says ‘because that’s where the light is’. Datasets shed light only on what’s in the dataset.”
We need to look beyond where the lamppost shines. This can be challenging, but dangerous if overlooked. Drugs and medical devices affect men and women differently, yet many drugs are never adequately tested in women and this lack of data has resulted in both fatal and life-changing consequences. One study showed neuroscientists used male mice over female mice on the basis that fluctuating hormones would render the results uninterpretable. Decisions like this prioritise the ease of data analysis over the accuracy of the results — in this case, for half the population. We can help address this imbalance by supporting organisations such as Women Count that are actively trying to close the data gap.
We also need to scrutinise any data being lit by the lampost. One way to do this is by filtering by sex, race, age, disability etc. in order to spot bias or weighting towards one particular group. A dataset of male COVID-19 deaths that disaggregates by location, race and age could tell you, for example, that in England, a black man is nearly four times more likely to die from the disease than a white man of a similar age. Without disaggregation, all you have is a total number of deaths.
As service designers, we have a duty to critically assess any data we are given by disaggregating and paying attention to any inequalities it may harbour: otherwise, we risk reinforcing existing biases within society in the products we build.
It’s not rocket science
Data is a fantastic resource, but it doesn’t replace the need to understand the human experience of our product or service. Feedback from real users is equally crucial.
How many tampons should you take for seven days in space? If you use tampons, you probably have a good idea of how many you’d need. If you don’t, how would you find this out? Maybe you’d ask someone who does?
A simple solution, but often overlooked. A team of male engineers from NASA famously suggested to Sally Rides, the first American female astronaut to go to space, that she take 100 tampons with her for the seven-day mission. That’s probably about five times more than you’d need.
In an industry where weight is meticulously budgeted and where miscalculations risk both people’s lives and millions of taxpayer’s money, it’s a relief that they asked her opinion and listened to her response: ‘No, that is not the right number’. (They also tied the strings of the tampons together so they wouldn’t float away… somehow thoughtful and thoughtless at the same time.)
Good questions open the conversation rather than lead it, and perhaps save you some embarrassment too. So let’s ask the right questions.
For example, focus group guru Naomi Henderson was set a task to discover what motivated a group of people to shop late at night. Instead of asking a leading question like: ‘Do you shop late at night because you didn’t get around to it during the day?’ she invited the room to share their stories: ‘Tell me about the last time you went to the store after 11:00pm’. This prompted a quiet girl in the corner to raise her hand and confess ‘I’d just smoked a joint and was looking for a ménage à trois: me, Ben, and Jerry’.
We all have the ability to discover these hidden gems that drive human behaviour if we allow ourselves to be vulnerable and admit that we might not know the answer. Coaxing stories that reveal the depths of human behaviour might take longer and more effort than reinforcing what’s already in plain sight. But if we actively facilitate honesty through open conversations, and welcome silence as an opportunity to listen rather than speak, we might discover genuine breakthroughs for a product or service.
In practical terms: we need to recruit diverse focus groups that reach beyond our targeted market. Then, within these focus groups, we need to encourage storytelling by asking open questions that don’t rob people of their narratives.
Stop, listen and question
Whether it’s a soap dispenser, a leaflet on heart attack symptoms, a facial recognition app, a rocket launch, or a late night Ben and Jerry’s, we all have a responsibility to design products that meet the needs of all users.
We need to take action– whether by scrutinising data we’ve been handed, or encouraging storytelling within focus groups — and pay attention to the realities experienced by others, particularly those different to ourselves.
Injustices in society are easier to address once they are statistically visible: by first raising awareness of this systemic data gap in design, we can begin to address it.
If we take the time to stop, listen and question, we might just shed light on exactly where the problems in our society lie. The keys are in our pocket after all.
By Juliet Sanderson — Service Designer, Deloitte Digital
