Do you really need more research?

Douglas, the user research dog, says you can never have too much!

Content warning: this blog talks about the cost-of-living crisis and the financial hardship people are experiencing.

The title of this blog might sound like I’m trying to talk myself out of a job. I’m not! It might also sound like the title of a blog just for researchers. It’s not.

This blog is an opportunity for reflection for anybody who does research or uses data and research findings in their work. So yes, it is for researchers. But it’s also for leaders, decision-makers, policymakers, press, academics, government officials and anybody else who uses data in their work.

It’s a reflection of how 2 things that have taken up a lot of my headspace for the last few months can be connected:

1. trauma-informed research practice
2. waste

And about asking ourselves if we really need more primary research using traditional data collection methods like interviews and surveys. Especially for ongoing topics that are causing distress or trauma to the people who are experiencing them.

I’ve used the cost-of-living crisis as an example, but this can be applied to everything.

Before the cost of living became a crisis

In early 2022, long before ‘cost-of-living crisis’ became a household term, we were doing a discovery project to identify gaps in the advice and support we offer disabled people. As part of this, we analysed the emails sent to the Scope helpline to identify information needs.

Analysing the emails offered us insights on:

• the naturally occurring information needs of disabled people
• the language people use to talk about problems they are facing
• what problems people come to Scope for help with

At this time, the cost of energy was increasing and lots of people were getting in touch with us for advice on paying for gas and electricity. But we noticed another pattern. A lot of people were asking for help to get food at the same time. At the time, the media were talking about a choice between heating or eating. But lots of disabled people were already going without gas, electricity AND food. There was no choice.

From the data in the emails we analysed, we were able to write a user story and prioritised creating the free food and food banks page in our workflow. We aim to provide content that helps people to solve problems. We tested the page with people who had accessed food banks in the past.

From testing, we learned that sometimes the biggest barrier to solving a problem is experiencing negative attitudes. Participants spoke about the emotional cost of having to retell their story over and over when seeking help. They also told us about the shame and judgement they felt when needing to access a food bank. Shame and judgement can lead to shame anxiety and trauma. Shame anxiety and trauma can make it hard for people to ask for help. And so the cycle continues…

Is now the right time to be asking questions?

Living in poverty and food insecurity can cause trauma responses and affect the physical and mental health of children and adults (The Robertson Trust). This is something we have heard from disabled people and their families. For example, people have told us that:

• symptoms of their condition are getting worse
• they are experiencing anxiety, depression and hopelessness
• they are trying to access mental health services but there are long waiting lists
• they are contacting Samaritans more frequently

Disabled people have told us that their current situation is “filling me with fear”. So should we be doing research with people who are experiencing trauma and fear? In our user research ethics policy, we say that we balance the safeguarding of participants with the need to include their voices. One of the ways we can do this is by being flexible about how we collect data.

Over the last 18 months we have been asked a few times what evidence and insights we have on the cost-of-living crisis. And if anything has changed in the last few months.

Some of this data has been collected for a specific purpose, such as to support a campaign. But most of it is data that we collect in our daily working. We aren’t asking people about their experience of the cost-of-living crisis, but people are telling us about it when they contact us for advice and support. We can still identify the problems and needs of disabled people during the cost-of-living crisis, but we aren’t risking causing further distress or harm to people.

For example, by using a survey to get feedback on Advice and support pages. And asking for consent to use the responses for research, marketing and campaigns.

What one person experiences as a sensitive or traumatic topic might be fine to talk about with somebody else. But trauma is trauma. Ethically, you would be very cautious when asking somebody living in an abusive home about their experience. You wouldn’t go back to those people on multiple occasions to ask them how much worse their situation is compared to last year. So why would you do it for other topics which can cause trauma responses?

We’ve tried to avoid going back to disabled people and asking them how much worse the cost-of-living crisis is this winter compared to last winter. We’ve been quite successful in this. But mainly because people are coming to us to tell us how much worse it is. We haven’t asked a question. But we still have the research findings.

What a waste!

The European Commission estimates that the cost of not having research data that is Findable, Accessible, Interoperable and Reusable (FAIR) is €10.2 billion. So, instead of doing more research, is there anything you could do to improve your ReOps (the people, mechanisms and strategies that support researchers to deliver research findings)?

Specifically, could you improve your:

• admin — to ask for consent to use data you are already collecting for research purposes
• tools — to collect and store data in a way that makes it accessible and findable
• data and knowledge management — to make the data you already have accessible to the people who need it and reusable for other projects

Could you make your research data FAIR? Ask yourself:

• What data do you already have on a topic?
• What do you already understand about a problem your audience are experiencing?
• How easy is it to find it?

For example, asking people for their consent to use the data they are giving you in a contact form, would let you use that data in your research. That data is the naturally expressed experiences of people, told in their own words. It would also avoid the emotional cost of people having to retell their story over and over.

Also, where do people naturally come into contact with your organisation? In our team, we talk about meeting people where they are to solve problems. This applies to research too. Where are the people coming into contact with your organisation? Can you meet them there? Are you already collecting data there?

For example, at Scope, we have:

• colleagues who speak to people every day
• helplines and services
• emails, online forums, social media accounts
• feedback forms

As well as reducing the costs of doing research and being better for the environment, it could mean that you only need to do more research when you actually need more research. And that means there is less risk of harm to people.

Final thoughts

So yes, in an ideal world, we would do research and collect data that is:

• representative
• replicable
• generalisable
• rigorous

But do any of those characteristics outweigh research that is ethical and humane? I don’t think so.

Does an event or a phenomenon that disproportionately affects a smaller group of people, but has devastating consequences on their lives, make it any less worthy of research? I don’t think so.

So maybe sometimes it’s ok if data tells a story about the experiences of a smaller group of people. And maybe it’s ok if you’ve analysed data that you’ve collected without asking anybody any questions. The findings are still valid. You should still use them.

In an earlier blog on participant wellbeing, Grace spoke about “do no harm” and if the avoidance of harm is the best we can aim for as researchers. She spoke about how we can make research flexible and give participants agency. But there is something else we can do. We can ask ourselves “do we really need more research?”

An important part of being a researcher is being reflective. This includes before doing things, as well as after. So please, when you’re planning your next research or planning to use research findings in your work, ask yourself:

• What data do we already have on this?
• Do we, or can we, collect data about this in our organisation’s existing work?
• Is asking people about this right now likely to cause them distress or harm?
• Does the benefit of having the data outweigh the costs to the participants?

After answering those questions, you might decide you do need to do more research. That’s ok.

But if there is an opportunity, be bold. Do research differently using non-traditional methods.

Or be even bolder, and don’t do research at all.