Big & Little Stories : Qual & Quant Data working together
Quant data is like a smoke signal that points to an interesting place. Qualitative research uncovers the story from the ground.
As a Design Researcher I perform qualitative research to advocate for users’ POV in our product design conversations. People sometimes ask:
You only spoke to 10 people, so, aren’t your findings just anecdotal?
If I chose 10 random people to talk about topics I chose on my own, yes, my findings would be anecdotal and quite biased. Instead I use quantitative insights culled from surveys, funnel tracking and other analytics to ensure I’m asking the right questions to the right people. The marriage of both Quantitative & Qualitative data ensures we’re having discussions that matter.
Quantitative Data = what / where / who of an issue.. surveys, tracking & other analytics
Qualitative Data = why / how things happen.. interviews, diary studies, and small batch deep product feedback
Good research uses a Quant | Qual sandwich.
I start with Quant to identify the important populations, subjects and /or moments in our funnel to focus on. Leveraging analytics, I know that people of a certain demographic are who I should be speaking to.
The next problem to solve is what to talk to them about. Are there 25 topics on the table to discuss? Put out a survey to that target population and figure out what the top 5 topics are. Scratching heads on a complex product? Work closely with analytics to see where interesting things are happening. Where is there major lift or drop off? Focus user discussions on those areas, or the subjects that impact those areas.
Sometimes it’s an open faced sandwich. Sometimes you need to follow up with another slice of Quant to validate anything you learned. For example, people kept bringing up X, but that’s only 7 people. Run a survey and see if that point of view scales. Better yet, see if you can find out what other factors correlate.
Inductive vs Deductive
When people think about research they most commonly think about Deductive studies where they have some ideas in mind and they want to deduce which are the stickiest. As with the example above: we have a list of features we’re thinking about and we’d like to run a survey to deduce which are the most important.
We may also use a qualitative evaluative test to deduce which UI approach is most effective at achieving our UX goals.
Before getting too attached to ideas to evaluate, it’s important to also include Inductive moments in your research that are open ended, making room for unanticipated concepts. This is where qualitative research — what you can learn by simply listening to people — becomes very powerful.
For example.. in user interviews I performed at Credit Karma I noticed an ongoing trend of people bringing up “stress spending” — spending they did to relieve stress that ultimately ended up becoming the source of stress, triggering an unending cycle. This unexpected concept surfaced because of open ended — or inductive — portion of interviews .
We needed to see if “stress spending” as an issue scaled before we could make a call on how to address it. I worked with my counterparts in quant research to deploy a survey about it. Not only did we find that it’s something many people experience, but we were able to inspire further surveys to understand more details. What were people stress spending on? How did different behaviors map to demographics? Our editorial team ended up writing a piece based on the study which was later covered by Forbes and NBC.
All because we took the time to listen and react.
This is all good and well, but you probably want to know how to use research to inspire a product direction. For one, don’t just send out a report. I’ve found that Design Thinking sprints can be very impactful. More on this in the New Year.