The 10 most common myths surrounding design research. Part 2
Welcome back! I hope you enjoyed the first part of this article. As a researcher and consultant I have been repeatedly hearing the same perplexities and comments, both from clients and colleagues, that are based on misconceptions around design research. So, I’ve decided to list them and try to answer them as best as I can. I hope you find them useful.
6. Your qualitative findings are not valid because your sample is “too small” and “not representative of the population”.
Ah, this one’s a classic. You’re showing the results of the research, and there is always someone with a frown that asks how can any of this be true and valid if you “only” interviewed 15 people. Then they throw in a few words about how that sample is also, “not statistically representative of the population”. Normally these people tend to be more familiar with quantitative research and what they are doing is they’re evaluating qualitative research using the same parameters that you would use to determine whether a quantitative research is valid.
A quantitative approach, unlike the qualitative approach we use in design research, is based on statistical representation. We use this approach when we want to generalize the results of the research to an entire population. We want to answer questions like what, when, where and how many. The sample is chosen deliberately to be representative and to avoid bias. Having a representative sample and a sufficiently large sample is a way to evaluate the robustness of the research.
However, in design research, most of the time we use a qualitative approach that borrows from ethnography and anthropology. The approach to sampling is very different because we have a different objective. We’re not aiming at quantifying a known phenomenon but at exploring, understanding and obtaining a rich and thick description of it. We want to understand the what, why and how. We’re studying complex issues like social or cultural dynamics, perceptions, experiences, etc. so, the approach to sampling has to be different. In anthropology they also call it structural sampling, because you’re attempting to understand a social structure and the relationships between the different actors in it.
Qualitative samples are not and should not be statistically significant. Rather, they are relevant.We want informants who will give us the most useful and rich information on the phenomenon we are trying to understand and to reach saturation of the discourse. The samples are small because data collection and analysis are slow and complex processes. Just analyzing the data might take from 2 to 5 times more time than what it took to collect it. That means a 2 hour interview will take between 4 and 20 hours to analyze. Large qualitative samples are just unfeasible in terms of time and resources. It is also not efficient — at some point you will reach saturation of the discourse and every new data point will add less and less new things.
Neither of these two research approaches can provide us with the whole picture on its own and neither is better than the other. They are complementary. It is possible to first send out an online survey and then once we have the results, run a small qualitative research in order to explore and understand the why of some of those results. The opposite is true as well — we could do the qualitative research first, and then quantify the findings to understand just how widespread or not they are.
7. Research is unnecessary, let’s just design something and put it in the users’ hands
This concern often comes from designers that oppose the Human-Centered Design approach. After all, they have been designing for decades successfully without ever asking a single user a thing and it has worked well for them. Why would they bother doing any research?
If you look at the history of design, you will understand where this concern comes from and why some designers still think like this. Back in the day when things were not done industrially, product design and manufacturing were done by the same person, often a craftsman (or woman). The objects were made either for people they knew directly or for themselves. Once the Industrial Revolution brought the concept of assembly lines, nobody was making an entire product by themselves anymore. This was the first time anyone needed a designer to produce a plan everybody needed to be aligned with. That was precisely how design was born.
Design is now entering areas like healthcare, banking, or insurance, where just fifteen years ago it was unthinkable to hire a designer. The digital products and services we are required to design today are even more complex and strategic than the physical products of the industrial era were. And the fact is, as designers, we don’t know everything. Most of us know almost nothing at all about these industries yet we need to design for them and for the people using them. In this context, research is not a luxury — it is essential. We need to understand what makes people tick, their needs, desires, barriers, etc. It goes almost without saying that research is fundamental because we are not designing for ourselves, we are not our users.
Design is already perceived as fluffy and irrelevant in many contexts, so backing up your design decisions with evidence is essential. Intuition can only get you so far. And locking yourself in a room arranging sticky notes on walls while pretending very hard to be the user of your product won’t do either.
Besides, for those who still insist on producing something to put in the users hands first “instead of wasting time on research”, I have bad news for you. That is research too. Research is not just explorative, it can be evaluative as well. This is where techniques like Usability Testing and A/B testing come handy. But at this stage of the process, you’re just looking to evaluate and refine the solution — the problem is well defined. That does not mean explorative research does not work or is not necessary at all. You’re just at a different moment in the design process and it’s not what you need.
8. Research is expensive.
When I hear this one I’m often baffled. What is more expensive? To spend a few thousand euros in a properly done research, or to spend millions of euros developing a product nobody wants?
Even if after doing the research you find out that nobody wants or needs your product, “no” is still an answer and it can potentially save you money because you won’t have to spend it in doing things nobody wants. By doing research we could find out what is it that people would need and why. We could redefine the problem and pivot the product idea. In early product development stages, changing or pivoting the product is relatively easy whereas as the process advances it will become more difficult and expensive to do so.
A good example of a product tackling a non-existing problem is the Segway. Back when it was about to be released, its creators were hailing it as the next big disruption in personal mobility. It was supposed to change everything — how we move, how cities are planned, the design of streets and parking spaces… And then it did not. There were many reasons why it did not work out, but the main one was it was not solving any problem. People had been moving from A to B using other means of transportation like bicycles, cars, motorcycles and their own two feet just fine. To make things worse, the Segway is expensive and in some regions it requires insurance, license and registration like any other motor vehicle. So, what are the advantages?
If you truly don’t have any money, you can always do guerrilla research. Ask friends or family members who are not familiar with your product or industry and interview them or show them your product. Some research is always better than no research! Guerrilla doesn’t give you the rigor and robustness of a proper research, but it is better than nothing and it can provide some insight into what other people think and feel when they have your product in their hands.
9. We do research to discover people’s needs.
Surprised I’m putting this as a myth? Let’s look at how this is phrased and read between the lines. If we discover people’s needs, then people know what they need, right? And all you have to do is to ask them, write down their answers and report them back. This phrase implies that all we do as researchers is document other people’s words and compile them into a nice infographic or report (we don’t, see myth number 1). The outcome of the research, whether it’s needs, expectations, barriers or customer journeys is the result of our analysis. It is not something that comes literally from the participants.
Furthermore, in Designit the strategic design consultancy where I work, , the biggest value research brings to the table is not “discovering” people’s needs, but the fact that it allows us to have a strategic dialogue with our clients using the research findings as the backbone of strategy and design decisions.
This is important because stakeholders in the client’s organization often have conflicting views on what the problems appear to be and on how they should be addressed. Organizations also have a natural tendency to be inward looking and focus on their own internal needs. Customers’ needs can become that common ground everybody agrees upon.
Furthermore, it will often be the case that a client comes with a briefing and says “I need an app”. But does she? Maybe she does not need an app at all. It could perfectly be the case that after the research results are in, we decide that an app is not what will fix the customers’ problems. Research lets you elevate the discourse, you’re not just executing a brief. You’re actively questioning and redefining the client’s problem.
10. Research always guarantees the success of our product or service
This assumption is, often, tacit and unspoken and it translates into pressure on the research team and this may even the reason why your team gets hired to do research in the first place. In the back of many people’s minds there is no other reason why they should pay for research if it’s not to have a “guarantee” of success.
But what is “success”? An increase in sales? An increase in traffic? Between the research results and any type of metric or KPI there’s quite a bit of distance. For starters, users won’t tell you what you need to design and whatever it is you need to design won’t design itself. You still need skilled designers and researchers that can translate the insights into actionable recommendations and requirements. Then, you need to define and design the solutions. After design comes implementation, and most of you know even better than I that this can be quite complex and challenging. As the project moves along, it is less and less clear whether any “success” or “failure” factors can be causally attributed to the quality of research results. Bringing a product or service to life is a team effort, and researchers are just a part of the team like anybody else.
Finally, whatever product or service we’re designing will not exist in a vacuum. It needs to compete in a market with a clear value proposition. The best research in the world cannot help you sell people things they don’t want or need.
So, can research “guarantee” success? No it can’t. And those who say they guarantee it are lying. Does it mean we shouldn’t do it? Heck, no! We should do it indeed, and embrace it as one of the factors that done right can guide you to success, one step at a time.
Thank you for reading! I look forward to your comments!
