Some Thoughts on Quantitative vs. Qualitative Research in UX Design

I’ve been thinking about the pros and cons of Qual vs. Quant lately. It’s hard not to, considering the company I work for “quantifies everything,” but many of my UX courses focus on in-person user interviews, fly-on-the-wall observations, and related ilk.

Both pursuits, obviously, have their place and their own strengths and weaknesses, but each method also has the same goal: to discover (and communicate) the truth.

In user experience design, research tends to focus on how users interact with a system, whether it’s one that has already been designed and requires refining or a system that you intend to improve with a new product of some sort. Either way, this is a crucial phase. Without good user research, you can’t actually design for the people who are going to use the product. You can only design based on your own experience + assumptions (and we all know what happens when you assume).

UX, it seems, tends to focus on qualitative research methods, at least when you’re designing something new. If you need deep understandings of the motivations, problems and past experiences that explain someone’s behavior, you probably won’t be happy with the results of quantitative methods. You won’t necessarily trust what surveys are telling you. You’re going to want to hear from those users directly, ask them questions, and actually observe what they’re doing.

The problem here is that insights from qualitative methods are so focused that they might not be applicable to most users. You might discover something really interesting that then sends you in the completely wrong direction.

On the flip side, data trends from web analytics, survey feedback, or other sources can give you a quick and trustworthy understanding of user behavior. This is especially helpful when trying to solve a UX problem with an existing system. If the metrics are significant, this approach allows reasonable assurance that you’re focusing on something that most users are experiencing (e.g. if 64% of people who visit a certain webpage will then use the search bar, that’s a strong indication that they aren’t finding what they need on that page).

However, it’s important to remember that even quantified data insights are still somewhat biased. Numbers give us comfort because they appear to communicate absolute truths, but every data report is prepared by people who choose how to display the information, what to emphasize, what not to show, etc. Data can also be just plain wrong if your source isn’t reliable.

Quantitative research seems to be the most appropriate method for business decisions, since decision makers are more apt to trust verifiable trends. Still, it can be difficult to get to the “why” just by looking at data. For example, you might know that an app’s users tend to log on at specific times of day, but you don’t really know why they do that until you get a chance to ask some of them. Sure, you can ask people using online forms, but the quality of that feedback pales in comparison to the intelligence gained from actually talking to them about it.

In an ideal world, we would be able to speak individually to a large segment of a user base, teasing out qualitative insights from a large enough number of people that the qualitative findings can then become quantifiable data covering a statistically weighty segment of the user population. Unfortunately, no one has the time or resources to accomplish this.

So I guess my conclusion here takes me back to where I started: both qualitative and quantitative research have a time and a place, with their own strengths and weaknesses. It just comes down to figuring out what your resources are, understanding the goals, and employing a healthy bit of skepticism when consuming either method’s results.