The forgotten participants
How one oversight made me question all my past research
As always, we found ourselves in a race against the clock to deliver research insights to our client.
A few participants were no-shows to their scheduled interviews, so we had to extend more time for recruiting. We couldn’t push back the deadline, so this left only one very short week for analysis.
I was pouring through transcripts late into the night, ever aware of the impending deadline. As the insights and narrative began to take shape, my co-founder Alex, took a look at my analysis.
To my surprise, he discovered that 2 participants I interviewed weren’t represented in any of my report’s insights.
I was baffled because I was certain my research was thorough and included every participant’s perspective.
The forgotten participants
I went back to re-read the transcripts, and Alex was right. Those 2 participants didn’t roll up into any of my insights. Not only that, but their interviews included a common theme that I missed in the chaos of analyzing everything else.
This hidden theme amongst two people I personally interviewed, was left on the cutting room floor. It was a little flame of an insight, that I had almost snuffed out.
I spent the next day refining this theme and working it into the research report. Not only was I able to present the initial insights I found, but I could showcase emerging use cases the client wasn’t aware of before. Our clients were thrilled to see the nuance and depth of our report.
Research saved! Hurray!
Questioning my past research
I felt great about the presentation. But as I reflected upon these two forgotten participants, I couldn’t help but wonder what this meant for my past research.
How many insights, or participants, did I leave on the cutting room floor?
In my last job, I captured a video clip of a user that was so persuasive that it altered our company’s entire roadmap. It turned Alex from a qualitative research skeptic to an advocate and became the topic for my podcast interview with Jared Spool.
There’s no doubt this past research drove the company to solve that user’s need, but now I wonder, were there any other users who I interviewed but left out?
Were there more insights that I missed in my process of analysis? If I overrepresented that user’s story, the whole company could have been making the wrong decisions based on my research.
This stirred up a deep fear that I’ve always had, which was that people that doubted the validity of qualitative research were right. It’s dangerous to rely on anecdotes in small sample studies.
A better way to share analysis
I’m thankful that this time around, Delve, the qualitative tool we built to analyze user interviews, made it easy to share my research. My coding created a catalog of how many users were impacted by each theme, and made it simple for Alex or I to evaluate how representative each theme was.
But I didn’t have these capabilities in the past.
My old process consisted of boards of post-it notes, and color coded spreadsheets that only made sense to me. In order to check which users rolled up into my insights, I would have to manually track every post it note, which would have taken too much time.
With the speed that my team moved, they would have taken action before my research was ready.
Additionally, there was no way for my teammates or stakeholders to investigate or actually see how my analysis took place. It took months — years of me educating them in order for them to trust what I do. With Delve, I can just show them.
Bringing transparency into a previously opaque process
I realized in this experience that Delve wasn’t just a tool to help research get done faster.
Delve brings transparency into a previously opaque research process. It allows the myriads of decisions and interpretations we make as researchers to be cataloged, traced, counted, and summarized.
Analysis in Delve consists of a series of interviews, which turn into codes, which get re-arranged into insights. This allows the whole process to be examined and investigated in a way that was not possible to do before.
Delve enables a level of rigor which we’ve previously only seen in academic research tools. But unlike those tools, Delve is accessible and easy enough to use that it enables researchers to keep up with tight deadlines.
We’re working hard to make sure that no participant is forgotten, and that every insight is captured. And thus, every decision made upon that research will be more sound.
Ready to do research with Delve? Join our waitlist here.
