5 principles for more accurate user testing
The outcomes from user testing have the potential to be extremely powerful in defining and informing the direction of a project.
However their potency and accuracy relies on the expertise of the facilitator who is responsible for collecting data that reflects the reactions of the participants, not their own personal agenda.
I have two beliefs, which may seem contradictory:
- User testing, and indeed Design Research more generally, should be held to a high quality and standard
- Design researchers should be up-skilling those with an interest in facilitating
So how might we create and maintain a high standard of user testing whilst supporting people with an interest in user testing to learn the craft?
I’ve been codifying an approach to up-skill designers, developers and product managers in the art of testing. Get in touch if you’d like to go through it.
In the meantime, here are some key principles.
1. Beware of Projecting
We are only interested in how the participant reacts and interacts with the tool. We are not interested in how they think other people might interact with it. Despite this, people will sometimes start to imagine how the product or service would be used by other people in their lives.
You might hear people say things like:
“I can imagine someone with less experience might find this useful.”
“I would do this, but people will generally do….”
“My mum wouldn’t use this.”
When you notice people projecting, steer the participant back towards sharing their own reactions by reminding them in a professional, non-scalding tone- “today I’m interested in what your reactions are so let’s stick to that. At the end of this session let’s leave some time to talk about how other people you know might use this product.”
2. Avoid leading questions
Leading questions are those which impose your own assumptions or experiences onto the person doing the test. Leading questions include or imply the desired answer.
Some examples of leading questions:
‘Why did you find that difficult?’
‘Did you find that confusing?’
Leading questions create a situation where the participant may not feel comfortable expressing an opinion or providing an answer that is counter to the lead.
By all means document your interpretation of the situation, after all, if you saw someone struggling to complete a task, then that’s a useful insight. However, to protect the accuracy of the data, do not include your interpretation of the situation in your line of questioning. Instead, remain neutral and try this:
‘What was your reaction to that?’
‘Which words would you use to describe that experience?’
3. Be polite but remember, you’re not there to make friends
It is often tempting when you’re conducting 1:1 qualitative research to spend time building a rapport with the participant. However, there is a very important reason why this should not be done. According to a study by the University of Massachusetts, 60% of people can’t hold a 10 minute conversation without lying at least once.
The reasons people lie are plentiful — they don’t want to offend you and they want to impress you are just two of those reasons.
If you spend time trying to create a friendship dynamic, you have shifted an important research paradigm. You have gone from two polite, consenting strangers having a conversation, to one where the participant is potentially manufacturing responses to please the researcher.
4. Remain neutral and avoid positive affirmations
Another basic human temptation is to positively reinforce the behaviour we desire. Witnessing someone complete a task in the intended way can be an exciting moment for a researcher. However, it’s really important to avoid leading and affirming statements like ‘great’ or ‘well done.’
This suggests to the participant that you approve of their behaviour and makes it difficult for them to have a contrary reaction at a different point in the test.
Use neutral phrases like ‘I see’ ‘OK’ ‘Sure’ and ‘Tell me more’
5. Reserve judgement
As researchers, we must be humane and create a comfortable environment for our participants, but we also must remain non-judgemental, after all we are not the arbiters of right or wrong.
Perhaps this is the one that takes the most practice — a simple, accidental eyebrow lift at the wrong time can immediately communicate your disdain — intended or otherwise.
Part of conducting qualitative research means you meet people with views and experiences of the world that are different from your own. It is not your job to educate or counsel. It is your job to research.
And what a great job it is.
