Distilling Insights From Diverse Interview Data is Hard
With the first two months of our capstone project complete, it was time to summarize our findings from several weeks of user research. Over the course of our initial discovery phrase, we had accumulated a staggering amount of data that gave us a window into several different verticals that could be a fit for Zensors. We ended up with:
- 17 half hour interviews and 2 one hour interviews
- 9 verticals
- Approximately 2–3 interviews per vertical
- A total of 21 participants from across the U.S.
All in all, we sent hundreds of requests for interviews in person, over the phone, via email, and online with platforms such as Reddit and Craigslist. I find it amazing that we were able to get a broad coverage of enough industries to provide us with a way forward!
So what did we do with our data? Our first attempt to analyze all of our data was to categorize our data according to our market criteria for the purpose of visualizing patterns in what interview participants told us. This means we needed a giant matrix to put our data into. Partially inspired by affinity diagramming, we created a row for each vertical and created columns using our criteria using tape to build a large diagram on the wall. The end result was a giant tic-tac-toe board that we could data points to. Step two of the process was analysis: determine which verticals and criteria had the most data points and take the relationship between those data points and criteria to gain insights.
Simple enough of a plan, but it turns out that wasn’t the best way to analyze the interviews of 21 different participants.
What we discovered using this method was that we were unable to draw insights from the data due to a significant amount of variation throughout our interview notes. Since we were just beginning to develop domain knowledge about the verticals we were interviewing, our interview questions spent time trying to understand the tasks, work flows, and issues our participants had at work. The result was a large number of references to tools and processes that — quite honestly — were too narrow to help us gain insight. More importantly, the physical act of putting notes together was incredibly time consuming, and by the time we had placed half of our data points on the board, we could hardly read individual notes anymore.
After almost two hours of attempting to wrangle the data as a group, there was no end in sight for the process. If our heat map was not providing meaningful information, then maybe it was time to shift gears and do analysis a different way. As a result, our team organized a second data analysis session and decided to go through each interview and map the notes to the market criteria.
This way we were synthesizing the data within the context of the interviewee and vertical, and our process didn’t involve taping thousands of data points on a wall.
After analyzing, commenting, and dissecting each interview, we came together to discuss verticals that had interviews with revealed possible use cases for Zensors. The end result? Out of 9 verticals, we found two that had potential: facilities management and industrial baking.
Whew! That was a long process of discovery research and synthesis.
Looking back, this experience was a fantastic lesson. Categorization of data requires much more highly structured interview data in order to be more effective, and understanding the methodology of conducting thorough interviews is something that takes a lot of practice. Keeping in mind the method of analysis can also help guide interviewers in making sure they tease out as much information as possible from the participant. Reflecting on this exercise, we’re now more ready to improve our research game for the upcoming weeks.
