Part Three: We owe it to our clients and users to do UX research well
Part Three (of Three)
Your research doesn’t prove anything
We can never “prove” that X leads to users feeling Y. The findings only support a theory, idea or hypothesis.
Unless you can interview or test absolutely every user you will never have researched all of the experiences of the customer group. It’s for this reason that most sciences, which rely on more rigorous and standardised methods than UX, generally don’t say their findings “prove” something to be true. We can never “prove” that X leads to users feeling Y. The findings only support a theory, idea or hypothesis. This brings me to another point.
Saturation is the true golden standard for quality research and content validity — it’s when you have ceased uncovering new themes or insights, and the project findings can be replicated.
8 isn’t the magic number
Eight participants should not be considered the golden number for all human centred design/UX research. I’m not talking about usability testing, I’m talking about methods where you’re asking a participant to share with you their experience of a product or service. There isn’t a precise number of people to interview, as with quantitative data which relies on statistics to determine the minimum number of participants. You’ll know you’ve interviewed enough people when you have reached ‘saturation’. Saturation is the true golden standard for quality research and content validity — it’s when you have ceased uncovering new themes or insights, and the project findings can be replicated. A smaller study will need less participants to reach saturation than a larger study, and more groups or segments will also increase the number of participants needed. Most times you won’t be able to achieve saturation, but you can at least encourage the client to be flexible with how many participants might be needed.
The irony of not being transparent or handing over the data
Something I’ve witnessed is that some practitioners don’t want to hand over the data to clients (although clients ask for it) at the end of a project. The reasons I’ve heard include “we never give it to them, it’s just industrial waste”, or “they never look at it again”, or “it makes more work for us if they don’t like what you did”, or to the client they say “my writing is so bad you won’t be able to understand what I wrote anyway”. At the end of the day, all of these reasons are excuses. And it looks like there is something to hide when there isn’t transparency. If you’ve done your research and understand that it’s not just about talking to people, if you’ve followed your method and plan, used methodologies that are reliable and facilitate validity, if you’ve been consistent throughout your process then you should trust your work and not have reason to hide it (including hiding data from the client). A good practitioner understands this, and can communicate the study limitations with confidence.
If you’ve been consistent throughout your process then you should trust your work and not have reason to hide it (including hiding data from the client). A good practitioner understands this, and can communicate the study limitations with confidence.
Again, I reiterate too that your research doesn’t prove anything. It simply provides support to a hypothesis or concept. If someone looks at your data through a different lens then yes, they might come to a different solution compared to you. Don’t shy away from that, be thankful that we all see the world differently — it could be a very boring place otherwise.
Annie has been a researcher since 2004. She started in clinical psychology research before moving to social research, and finally translational research with mental health users. She loves to challenge the status quo of research and practice. She’s passionate about improving the way we work because she has seen the effects that valid and reliable methodologies, and best practice can have on creating change.
