Three biases to avoid in usability testing

User testing is a hot topic yet few people have mastered the art!

The aim of this post is to share one (maybe three) bits of research and best practice that you can apply to creating better online experiences.

Two questions before we get into the research…

1. Do you conduct user testing, but ask leading questions?

2. Do you collect insights but don’t know how to translate them into action?

If you answered yes to both of these questions (or even one of them) — this article is for you.

Research

1. Confirmation bias — it’s the tendency to interpret information in a way that confirms a persons pre-existing beliefs. It’s that bias that goes a bit like this in your mind, “Ohh that’s what the user meant by that”. Instead the question should be, “What did the user actually say verbatim?” or even better “How can this behaviour be measured? — User failed to login in twice”
2. The Hawthorne Effect — it’s when users behave differently because they know they are being watched. Basically they stumbled across this effect while looking for the optimal environmental conditions for productivity in the 30’s. In the studies they found that higher productivity was due to people being observed and not changes in the environment. You can read more about this research here. This means the best user research is done when users don’t actually know they are being observed. For example, if you have a website or app you can track specific user behaviour using software like Segment and Mode Analytics.
3. Social Desirability Bias— the tendency for users to answer questions in a way they think you want them to respond. This can mean users tell you one thing, but actually do another. One way to try and get around this is to ensure you ask thought provoking questions. You know when you ask someone a question and they squint their eyes and look at you like their mind is processing as fast as an intel processor?? Those ones. More on how to do this in my next post — stay tuned!

How can you overcome these biases?

Short answer. You can’t, BUT you can take steps to minimise their effects.

Step 1. Awareness — Read about these these biases and the many more that exist

Step 2. Identify — Try identifying when these biases appear for you and if you can notice them in others.

Step 3. Practice, practice, practice.

Follow Melbourne’s User Experience and Psychology Community @uxandpsych — sharing research on what makes people tick to create better online experiences. We also run a MeetUp once a month in Melbourne!