Illustration by Raphael Sousa

Number5 and Design: Part Two, Mixed Methods

Number5 and Design is our three-part series on the role of data within strategic design research. In our second installment, we will outline scenarios that call for a mixed methods approach and which frameworks to use. Before diving in, be sure to check out our introduction to integrating quantitative techniques with strategic design!

By Nikolas Black, Diego Jimenez, Miguel Bello, and Raphael Sousa, our designers in Medellín

Mixed methods research combines both qualitative and quantitative approaches to increase the depth and accuracy of design user research.These methods are rising in popularity in many fields, such as psychology, marketing, and public policy, and they have untapped potential for design. In this article, we’ll outline when using mixed methods is appropriate, the different frameworks to structure research, and the importance of using these techniques in strategic design.

When should I use mixed methods?

The first question to ask is: why do you want to use mixed methods? All types of research have their strengths and weaknesses and mixed methods are no different. We need a reason to justify the additional complexity of a new type of data. Fortunately, there are plenty:

• You’ve discovered a new phenomenon when running your numbers, but you can’t get the full story from the data — Qualitative information will provide a better understanding of why this weird thing has occurred, and Designit provides these insights everyday. Maybe you didn’t put a name to it, but this is a mixed method! Clients often approach us to find the “why” through numbers.

For example, Bancolombia realized that many Colombians were skeptical of traditional banks and avoided using them, but they didn’t know why. Our team at Designit used qualitative research to discover the reasons behind their challenge, ultimately designing a solution to engage their potential customers. We created Nequi, an all-digital bank that provides financial services without the stigma and bloat of traditional banks.

• You’ve found a few different ways to measure a variable, but those methods were used in other contexts, so you’re not sure they’re 100% suitable — Mixed methods can lend important context to improve their performance.

For example, imagine you’re Uber. You have a rating system for drivers from 1–5 stars. In North America, where the app was developed, your drivers have an average of 4.9 out of 5. That’s pretty good right! When you expand to Europe, those drivers have an average of 3.8 out of 5 stars. That’s terrible! Conclusion: European drivers are worse.

Does this conclusion make sense? Nope. The real story is that Europeans tend to be more conservative when assigning ratings, while North Americans tend to select 5 out of 5 by default. Mixed methods are key here: the ratings ensure Uber’s drivers perform well, and understanding context is key to adapting ratings between cultures.

• You’ve performed a few interviews and discovered that people find conflicting factors to be important –So which factor (characteristics, aspects, attributes, etc) should the new product or service focus on? Mixed methods can tell you which of these factors equates to a larger market share and whether a design will work.

Mixed methods frameworks

It’s always best to work within a framework to get the most out of your data. There are many different approaches to mixed methods. These four strategies synergize different types of data to enrich almost any study:

1. Triangulation [Qual and Quant] — Concurrent collection of both types of data combined into a single result.

Triangulation can be used when comparing different designs. Let’s say you have two different designs for a feature to add to a webpage. You can conduct focus groups and in-person demos, while simultaneously releasing both designs to a subset of users to measure KPIs, such as click rate or user spend. Blending these insights dramatically improves our ability to choose the best design.

1. Embedded [Qual → Quant → Qual (or the inverse)] — One form of data complements the other. For example, using qualitative data to add a story to numbers, or quantitative data to reinforce the significance of qualitative findings.

Embedded research can be used to translate your work into your client’s language. C-suite executives are often looking for how our work will impact their metrics, especially the bottom line. Statistical techniques can help translate the core work of designers into the numbers these managers are looking for. Showing the value of your work will never be easier.

1. Explanatory sequential [Quant → Qual] — Perform some sort of quantitative analysis followed by qualitative data to explain why the results were what they were.

Explanatory sequential methods allow us to identify new opportunities from data and then explore them qualitatively. This adds a powerful new dimension to our research process. When the client approaches us with a need, we can use their numbers to better understand their project, exposing new areas to probe with our usual techniques. Designing from this research may lead in directions we wouldn’t have discovered otherwise.

1. Exploratory sequential [Qual → Quant] — When trying to perform quantitative analysis, the researcher needs to know which variables to measure. These variables may come from an academic model or previous experience. When exploring a new field, qualitative analysis can flush out the right variables to measure and indicate how they may interact.

Exploratory sequential studies allow quantitative research on a subject that hasn’t been looked at before. Organizations often work with Designit precisely because they want our perspective on a new problem they’re facing. Our Future themes is a perfect example of where exploratory sequential designs can be used. Qualitative data defines and frames these new fields, such as trusting invisibility, while quantitative techniques provide rigor and enhance the credibility of our ideas.

Get ahead of the curve

Mixed methods is a huge field encompassing many different disciplines — way beyond the scope of our humble series. If you’re hankering for a deeper exploration, Dr. John Creswell from the University of Nebraska-Lincoln has an illuminating presentation that we’ve referenced here. It contains several resources that will help you learn about new techniques for your research.

We’ve covered a lot of ground in parts one and two of Number5 and Design! Join us for our third and final piece to learn how our teams at Designit integrate numbers with design.