Businesses Who Say ‘We Do The Research’ Can Have Some Big Flaws With Their Methodology

Better conclusions help deliver the truth.

Over the past few years ‘user experience’ has been embraced and adopted by companies who have taken on ‘UX’, more specifically UX research, and incorporated this into their marketing and strategies.

This is fantastic, however, whilst the implementation of UX research appears positive, there are some pitfalls to this uptake.

The biggest challenge the UX industry faces is one that has to be addressed. Some companies claim they ‘do user research,’ but fail to consider the basics and best practices when it comes to conducting this.

Let me highlight some shortcomings and solutions.

From Experience

In my experience, there are common misconceptions about what user research consists of, specifically when it comes to usability testing, and how this should be conducted.

Usability testing seeks to evaluate and uncover issues with websites and other online services.

This form of testing usually involves watching, monitoring and listening to users as they experience or navigate their way online. The most common method I hear from those that ‘do user research’ is asking users what they think?

Now, don’t get me wrong, I highly value users’ opinions and their thoughts surrounding their online experience. This can gather a more in-depth understanding of how users are thinking and feeling.

Users’ thoughts and opinions do have a significant place in user research, just not as much in usability testing. I have been privy to many examples where designers and other stakeholders take opinions and comments as the absolute truth and place significant value on the words that are spoken, rather than observing what is happening right in front of them, in the moment.

When I first started out in UX (before this I had spent five years studying psychology, from an undergraduate degree through to a master’s degree), I got to see firsthand how stakeholders can rely too heavily on users’ opinions. This was particularly relevant for clients looking to gain validation over their own designs.

Whilst it is encouraging for them to hear that people approved the designs and felt it was easy to use, what we observed was far from the truth. I have often heard users say a website was “easy to use” and yet I would have just witnessed them having difficulty navigating around a website.

This represents just one of many examples as to why you should not place all your value on users’ comments.

Strength and a truer picture is to shift this placement of value to the observations we make when watching them use websites, online services and apps.

How Can We Ensure Our Methods Are Valid?

The easiest way to improve the methodology is to understand the significance of actions rather than words when it comes to usability testing.

What users say versus what they do is extremely different. Fundamentally usability testing should look to mitigate against any bias a researcher or designer may have about how an online service/ website etc. will perform. This gains reliable and valid findings that can inform how these online experiences could be improved.

By using observational methods, researchers can see firsthand what works and what does not when it comes to an online experience. Designing a test that includes tasks and scenarios to users to work through first will reveal and deduce far more useful insights into how the site is performing, giving you a far more true to real-life glimpse into their natural behaviour.

Any questions or probes surrounding their thoughts should be reserved towards the end of testing, focusing on the things that they found useful but also the things they found frustrating or confusing.

My Final Thoughts

By utilising more observational methodologies we gain a much more valid insight into the users’ experience of online services and websites.

Even though we should not disregard the opinions of users we should first watch what they do before gathering their thoughts and discussing their feelings.

There is so much that can lead a business to a place that isn’t necessarily true. When research works it can have a huge impact on the success of products, service and interactions. It is our responsibility and obligation to reflect how people interact that sets the precedent for the people who will engage in the future.