Number5 and Design: Part Three, How we integrate numbers with design
Welcome back for the third and final part of Number5 and Design! In this series, we’re making the case for a hybrid research approach in strategic design: mixed methods. Mixed methods combines the complementary strengths of traditional design methodology and quantitative research to provide a well-rounded picture of your user base. For our final installment, we’re sharing a case study featuring the techniques we’ve previously covered. If you’re new here, we recommend riffling through part one and part two before jumping in.
By Nikolas Black and Raphael Sousa, our designers in Medellín
In our client work, we’ve witnessed the power of mixed methods design. With one client, we recognized that a range of approaches would be necessary to meet their needs. We identified three categories: fully understanding user behavior, answering business questions in their language, and iterating our designs quickly and consistently.
Here’s how we used quantitative techniques to bring value and affect real change for our client.
Understanding user behavior
Using either qualitative or quantitative methods wasn’t going to give us a complete understanding of user behavior. Qualitative analysis provides rich insights into why users take actions, while quantitative techniques support those findings and answer other key questions, particularly when evaluating the performance of our iterations.
We used analytics to identify which of our qualitative personas best fit potential customers as they began onboarding. This allowed us to follow their journey through the entire process, observing where different types of people dropped out, struggled, or successfully navigated the full sequence of the funnel. We could target our iterations to groups of leads to ensure an optimized experience. Without the quantitative bridge from personas to real users, none of these insights would have been accessible on such a large scale.
Once we categorized the leads, we analyzed their experience of the client’s website and application. We created heatmaps and recorded users’ interactions and data, such as click rate and the time spent on tasks. Each of these factors was then broken down by user profiles to understand how best to optimize our services for individual users. When we made site-wide design decisions, this data allowed us to understand what was best for the most customers. If we had solely used qualitative methods, the scale and depth of these insights would have taken months, instead of days, to acquire.
Translating impact into numbers and KPIs
Like many companies, our client needed numbers to back up our claims. We defined over 20 KPIs fed by over 60 metrics to calculate the impact of our decisions into a common language. These KPIs measured user experience, process efficiency, technology effectiveness, and business factors. We considered customer churn rate and the adoption percentage of both B2B and B2C leads. KPIs were valuable because they were consistent and measurable across time and business functions. We knew straightaway how well our features were working, and we could showcase our impact at every touchpoint.
Iterating with speed and consistency; growth hacking approach
The client targeted an extremely competitive cost of acquisition for B2C customers, far below the industry average. Based in all learnings and market metrics, we created a dashboard incorporating both qualitative and quantitative data to forecast the impact of our strategies on the cost of acquisition. Our key variables were extracted from each step of the sales journey, which we had designed. Using the dashboard, we identified focus areas for design elements and predicted whether each iteration would meet the mark. Once the methodology to collect and analyze the data was established, we could reuse it across many different contexts for real-time feedback and implement further experiments to foment growth and learnings in many aspects.
Lessons learned
Integrating quantitative techniques wasn’t entirely smooth sailing. Along the way, we learned many lessons, but the most important insight was to escape functional silos as soon as possible. Our most important lesson was to escape functional silos as soon as possible. When we began, we had a clear separation between the quants and the wider design team. It turns out this is a wider industry trend. A report by McKinsey on the value of design for business found that the companies with the strongest financial performance could break down the functional silos between designers and other business functions. Once the full team realized the value of quantitative techniques, we formed two structured strategy arms to promote synergy between the different approaches: one concentrated on focus groups, the other on data and massification. Integration between the arms was ensured through research questions that were answerable to the fullest through a combination of techniques.
Impact
We dramatically improved the strength of our conclusions with our mixed methods approach. Displaying our impact on KPIs helped us wield influence with the client’s steering committee, which made decisions critical to the success of our project. They adopted most of our recommendations because they shared our conviction that Designit’s strategy would be successful. The scale of our data also enabled us to achieve the massification necessary to meet all our targets. Our approach pushed the client’s market-share goals higher by a full order of magnitude, while providing the data to have confidence that our project would be successful.
How can strategic designers use quantitative research to supplement qualitative information? Why and when is this approach valuable? How can I implement it in my own work? We hope our series has provided some answers and an introduction to this broad and fascinating field. If you can’t find the answers to your burning questions in parts one and two, feel free to drop us a question in the comments below.
