Centring your service user in data collection
By David Ainsworth, freelance journalist. This has been cross-posted from the Data4Good Festival website as part of a series of posts written by speakers at the 2021 Data4Good Festival.
Charities and their service users frequently face a power imbalance. In theory, the charity exists to make life better for service users. But compared to each individual, a charity can seem to be a powerful, wealthy institution. The people that a charity helps often lack a voice and power in society. It may be difficult for them to make their wants and needs heard.
When charities gather data about their service users, they face the problem of how to centre the data around the service user. How do charities avoid colouring the data with their own biases? How do they ensure that the individuals feel ownership of the process and the results? And how do they ensure that the data helps deliver what users need?
Show respect
The first step is to start with basic respect, says Akil Benjamin, strategy director at Comuzi, a design studio which works extensively on social issues.
“People already understand that you need their data,” he says. “They’re happy to share it, so long as they feel you are going to use it well.”
Similarly, it is good practice to tell people what you are going to do with their data, says Ben Proctor, data and digital innovation director at Data Orchard, a social enterprise that provides data support to the voluntary sector and civil society.
“Be aware there is a power relationship,” he says. “Someone decides what data is being collected. The people who decide what’s being collected have more power than the people it’s being collected about.”
Try to minimise the power imbalance, he suggests, while being aware you can never eliminate it.
This approach is likely to make people happier when they interact with you, he says. But it will also yield better results. The more you involve service users in designing your interventions, the more effective those interventions are likely to be.
“Universities have form for not doing this well,” says Christine Goodall, network coordinator at HEAR, a network of equalities groups. “Researchers go into communities and they ask questions and nine months later the research appears behind a paywall and the community never sees them. The results aren’t ever used to benefit them.
“We’re keen that charities don’t do that. Only collect data from people if you plan to help them with it. And go back to those people and see if they’re happy with what you’ve done. Also, be clear about not just what you will do with the data, but who you are sharing it with.”
Benjamin warns that asking someone for information is not a neutral act. There is an emotional and social burden attached to asking and answering questions. People feel obligated to engage, which uses up mental energy that is then unavailable for other things.
“Ask people how they want to be engaged,” he says. “It won’t be the same for everyone. Some people want to be active participants. Others want to be informed. Others don’t want to be engaged with at all. Unless you meet people in a way that matches their lives, you can’t co-produce and co-create.”
Data vs individuality
One problem with using data, says Proctor, is that it involves simplifying the world.
“The world is incredibly complex and when you measure it you are reducing that complexity,” he says. “That’s intrinsic to the process of data gathering. There’s a model that you’re using to relate to the world. But the model is not the reality.
“All models are wrong, but some models are useful.”
For the charity sector, this is a continual challenge. One of the strengths of voluntary sector organisations is their ability to treat people as individuals, and provide holistic, whole-person support.
“The problem with categories is that they might be useful to you, but are they useful to the people you’re talking to?” says Goodall.
“So we are looking at whether people can self-identify when talking about things like race, gender and sexuality. That can cause problems if you are trying to do technical comparisons, but there are ways to get around that. We’ve tried it several times at events and got interesting results. One person just wrote ‘I am me’ on the form.
“People of mixed race and parentage say they really feel the benefit of getting to say something more detailed.”
She says that on topics including religion, sexual orientation and gender, people want to make more nuanced comments than the categories allow. This is more time-consuming but also more detailed.
“In order to understand and deliver charity services you need to understand the whole person and how different aspects of that person affect their lives,” she says. “People face multiple different barriers at once.
“That means you need diversity in your own team. Your people are more likely to understand your service users if they have similar experiences.”
“Remember, it’s not about you,” says Sam Rhynas, head of data at Effini, who works with a number of charities to help them use data better.
“Often charities gather data based on what you personally need, and how you view the issue. We ask questions based on how we divide and segment activities — who delivers them, or where, or the financial code they come under. Frame them in a way which makes sense to the person who’s answering the question.”
She says the process of gathering quantitative data encourages the creation of categories and archetypes — a small number of stereotypical beneficiaries. But this may be at odds with what is needed.
“People get focused on the fact that they’re providing a service for people exactly like this,” says Goodall. “But there aren’t any people exactly like that.
“Once you realise that, you can get caught up in thinking you have to help everyone, but of course you can’t. So you need that balance. You need to understand what characteristics are really core to the people you’re helping and the service you’re offering.”
One way to tackle this is with qualitative data, says Proctor.
“Qualitative data is necessary to find out what people want,” he says. “You have to ask people what they need. You have to talk to people in detail, for a long time, to really understand.
“Quantitative data is good for checking whether you’re getting the results you expect, and that’s extremely important. In a charity you use qualitative data to build a theory of change, and then you use quantitative data to check whether it works as expected.”
Assumptions and bias
Once data is gathered, it has to be interpreted and analysed in order to lead to useful action. At this point, says Proctor, it is important to be aware of your own limitations and susceptibility to bias.
“All human beings are terrible at data analysis,” he says. “The difference is just that some of us realise it. So try not to be deterministic. Don’t say ‘What is the data telling me?’ but ‘What questions is the data raising?’ Focus on opening up your thinking, not closing it down. Ask questions, don’t seek answers. And don’t just look at your own data set. Cross check with other sources.
“Look for good stories in the data. Look for things that are helpful to you and your colleagues. But don’t get sucked into thinking this is the truth. Don’t analyse data alone. Get someone else to look at it separately. Make sure that there is independent analysis and if necessary, ask someone to deliberately take a different point of view.”
There are a whole range of ways that data collection and data interpretation can mislead, says Rhynas.
“One of the most common problems is that no one actually asked for data at all,” she says. “There are lots of reasons for that. When it comes to gender and sexuality, for example, it makes some people so uncomfortable that they just guess.
“If you do ask, it’s very common to not ask the right people. If you gather data from people you’re already in contact with, for example, that’s a perfect place to introduce bias. Because the people you may want to know about may be the people who aren’t using your service. The gaps are often the most interesting bits.”
“Not asking all the people you should is one of the most common problems,” agrees Proctor. “The book Invisible Women is just a litany of things that would never have happened if someone had just asked women.”
Even if you do gather data from the right people, Rhynas says, the way that you ask is likely to lead to problems.
“If you just ask everyone, you will get responses from the enthusiastic survey filler-inners,” she says. “They may not be representative of the population you serve. Or you may start hitting problems with digital literacy. Some sections of your target group might not have a computer. Or they might not have unlimited data. They are less likely to respond.
“We can be quite scornful of using paper, but that can be a good method of hearing from people who don’t otherwise give you their feedback. If you have a day centre, you might be better off walking around and asking people to fill in a form.”
She lists any number of types of bias which can creep in when collecting data. Issues such as where the questions are asked, and when, and by whom, can all affect what answers you get. Or even something as simple as the fact that if you ask questions where there is no strong preference, the first option on the list can get ticked a lot more.
“And your own culture can make a big difference,” she says. “It can lead you to make assumptions which you don’t know you’re making.”
Conclusion
All of this means there are several steps to take when centering the user, most of which are simple to describe, but sometimes hard to do. Show respect when collecting information. Talk to people in depth and recognise that users are individuals. Ask them how they would like to engage with you, and design your processes to make that happen. Think about the mechanisms and frequency of collection, and how that may lead to bias. Be aware also of your own biases, and minimise them by employing people with different perspectives.
And share the results with your users, and empower them to tell you whether you are drawing the right conclusions, and delivering what is really needed.
If you think that DataKind UK could support your organisation with its data question(s), take a look at the free support we offer here.
If you’re interested in joining DataKind UK as a volunteer, have a read of our Volunteering page.
