USER RESEARCH
Winning the battle with bias
How to make sure your research is gold star standard
Working with and developing technology-based solutions for clients is one of the most rewarding and challenging experiences I’ve had in my career. As part of the planning activities in the discovery phase of these projects, user research is an absolute must.
Conducting research activities and synthesising findings are key activities to get right when developing a solution that has good user experience (UX). Solutions should follow a user-centred approach to make sure the design is UX approved.
To do user research activities the right way it is important to know the ways it can go wrong. Even experienced researchers are not immune from issues that can arise when conducting user research.
One of the big issues is research bias. When planning user research activities, you must be aware of types of bias that can subconsciously set the tone for research, and actively avoid them.
Many of these forms of bias can be mitigated or avoided by adopting best practice research techniques. This includes asking open questions, being open-minded, and avoiding making any assumptions or judgements of your research participants before the activity.
Here are some common forms of bias that can appear in UX user research and how you can avoid them.
1. The framing effect
The framing effect is one of the most common forms of bias that can occur in any type of user research. The concept of the framing effect is that people react differently to the same information dependant on how it is presented to them.
An example of this would be asking your users ‘what did you prefer or dislike about this product?’. A question asking what they prefer or dislike about the product can cause the user to only focus on only these two aspects of the product or service. Users are far more likely to give feedback framed around a specific question and could leave out important insights that contributed to their experience with the product.
To avoid this, user researchers should adopt a neutral approach to questioning. Questions can include ‘can you describe your experience with this product?’ and ‘how did you feel while using this product?’. These questions do not frame how a user should respond and allows them to freely express their experiences.
2. Social desirability bias
As humans, we want to present ourselves and our actions in a way that makes us look desirable amongst our peers wherever possible. This trait can arise when collecting evidence from user research activities and present as a form of bias.
This subconscious need to be perceived in a certain light can lead users to report feelings or experiences that will make them appear a particular way, rather than recounting their honest experiences. This kind of bias can arise when conducting all forms of research but can be particularly common when it comes to researching or testing a user’s experience with technology solutions.
An example of social desirability bias is a user reporting that they had no issues navigating new technology because they wanted to appear technology savvy.
To avoid social desirability bias, researchers should ask follow-up questions, or frame the question in a way that could to boost the user’s social desirability without highlighting any perceived potential weaknesses or difficulties.
3. Fundamental attribution error
This type of bias is typically associated with technology-based user research. Fundamental attribution error is when users blame themselves for not being able to understand technology or interact with a service.
A key identifier for fundamental attribution error is when a user describes their experience with technology and says they ‘made a mistake’ while navigating, using or testing it.
To avoid this type of bias occurring, ensure you are complementing your interview or focus group with observations and other types of research methods. It is somewhat common among engineers and product developers to shift any usability issues to blame the user and correct them on how they should be using it.
Researchers should conduct complementary forms of research methods in addition to one-on-one interviews or focus groups.
4. Confirmation Bias
Confirmation bias is another very common form of bias. Unfortunately, it is also very difficult to correct.
Confirmation bias occurs when a researcher displays data that confirms or supports their hypothesis and omits data that may challenge or reject their existing beliefs. This time, the onus is on the researcher. It can be very hard to accept and validate the presence of data that goes against your hypothesis, but in not doing so you are falling into the confirmation bias trap. Sometimes this act is deliberate, but other times it can creep in subconsciously.
An example of confirmation bias is a researcher discrediting feedback from a user because it may feel obvious or not important to them but is still valid feedback from users.
The best way to avoid confirmation bias is to ask questions about a user’s response, especially if it supports the researcher’s hypothesis.
Make sure you are acknowledging and validating all findings, whether they support your initial hypothesis or otherwise.
5. Sunk cost fallacy
Similar to confirmation bias, this type of bias arises from researcher action rather than participant feedback.
Sunk cost fallacy is another common form of bias. It follows the idea that the decisions made by humans are influenced by our emotional investment we have in them. It argues the more we invest in a decision the harder it is to desert it.
An example of sunk cost fallacy occurring in UX research is a researcher obsessing over their findings and becoming disconnected from the bigger picture. This can lead to the researcher following findings that may have little or no relevance to the actual research subject.
The best way to avoid this kind of bias is for researchers to adopt a mindset where they are at peace with the idea that some of their ideas or concepts may fail, and to learn and move on from this. Easier said than done.
6. Implicit bias
Implicit bias can be compared with stereotyping in popular culture. It involves researchers subconsciously associating certain attitudes or stereotypes to user demographics.
In UX research, implicit bias could look like predicting how someone from a particular demographic will interact with your product. Implicit bias is a hard one to overcome, as it has been embedded in our consciousness and commonly observed in wider society.
Implicit bias can be minimised by encouraging researchers to maintain a professional relationship with their participants, and to try not to assume how they are going to respond to various questions or testing based on the little you may know about them
