How to combat confirmation bias when testing your product
“Believing is seeing.. once we have a belief, we see the information that will confirm that belief, and we stop seeing what we don’t want to see. We want to see the evidence that confirms our beliefs, and we want to forget anything that is dissonant or discrepant.”
Confirmation bias is a form of bias that draws on a persons preformed beliefs about their view of the world. Being a well established part of the person, confirmation bias needs to be fought when approaching user research and usability testing. Looking at this issue through the eyes of a designer, the best way to overcome it is simply to not test your own designs. As designers, we become very close to our work due to the many painstaking hours spent adapting and evolving it and believing it has truly solved the problem.
During a usability testing session the design or functionality issues within the product may not be uncovered due to the facilitator unknowingly leading the participant in a certain direction and not seeing when problems are present.
Having conducted usability testing sessions with people present who are very close to the product, we have seen confirmation bias first hand. One session included a manager who was meant to be a silent observer, but as they had been working so closely with the team building the product and actively contributing to training materials and business process, they could not help but interject and lead the user. Whenever the participant struggled with a part of the product, the manager would tell them where to click next and even began explaining the process behind it.
Why does this happen?
To put it simply, you want the product to succeed! The more you work on something, the closer and more attached you become to it. You may not want to realise that the designs you’ve been working so hard to perfect and the interactions you believe are innovative and intuitive aren’t quite as user friendly as you expected. The closer you are to a design/product/service, the more invested and attached you become, which leads to you wanting to see it succeed. This can be seen in a lot of startups who may not have tested their concepts early on and with an impartial approach which ultimately leads to the detriment of the product.
So how can you remain impartial and combat confirmation bias?
Do not use leading questions when usability testing. One way to address this if it does happen is to make sure the notetaker has been briefed prior to the session to write down when leading questions have occurred.
Preparation is key.
Prior to any research or testing session with a user, mentally prepare yourself and open yourself up to new information.
Be empathetic.
Personally, empathy seems like the greatest tool to combat confirmation bias. By placing yourself in the participants shoes, you may become more open to seeing the bad as well as the good.
If you’d like someone to independently usability test your designs, feel free to get in touch.
