Data is Data: When Combining Different Styles of Interviews Makes Sense

In our last post, we covered a bit about how the decidedly un-UX-like nature of our project meant that we had to take a different approach to research that traditional design projects have to. I certainly agree with Marvin in that academic research — primarily that which comes from technical research in computer science — often seeks to build something before truly looking for the need for that something in the first place.

Although a product-out mentality might seem strange in the world of UX, there are a number of success stories that have claimed innovation based not on what the user says, but rather what the internal teams were able to come up with. Apple under Steve Jobs can be considered a product-out company: a single visionary leading a team of talented employees creates innovative, ground-breaking products that fundamentally change the technology landscape. If it worked for Apple, why don’t we emphasize this type of thinking in UX?

The answer’s simple. Product-out frameworks can often make mistakes because the engineers and designers are out of touch with customers. Scores of companies have been accused of creating products that resonated poorly with its market. Rather than go into detail, I’ll make use of a quote often attributed to Henry Ford (albeit without proof):

“If I had asked people what they wanted, they would have said faster horses.”

(For those interested, Harvard Business Review has an excellent article that not only comments on the lack of evidence we have that Henry Ford stated this at all, but also explores the history of the early automobile business.)

Even if the quote is misattributed, the words themselves are valuable because the sentence encapsulates that product-out isn’t necessarily bad. In order to accomplish the goal of product-market fit while bringing in a distinct flavor of , we decided to adopt a hybrid methodology when it comes to exploratory user research: contextual interviews and customer discovery interviews.

Contextual interviews are a powerful tool based on traditional anthropological methods in research in which an interviewer visits the interviewee in an appropriate environment for the product idea that is being tested. Researchers try to go in with as few assumptions about the interviewee as possible and build an understanding of the her through from the ground up. With this method, the context helps inspire questions during the interview process and also provides additional information to the researcher when it comes to interpreting an interviewee’s words. These interviews can take an hour or longer.

Customer discovery interviews are short fifteen minute interviews conducted over phone. Their goal is not to learn as much as possible about an interview participant in order to build an understanding of the participant, but rather to ask her a number of questions that test specific hypotheses about how the product might solve her needs.

Our take on customer discovery is based on talks with Kit Needham of Project Olympus, a CMU-based incubator attached to the Swartz Center for Entrepreneurship. She recommends at least 100 interviews of 15 minutes each in order to understand product-market fit. These questions are pinpoint:

What is the pain/problem?

Who is having the pain/problem?

How bad is the pain or…

How are the “who” currently dealing with it?

How is that solution working for them?

While we’re not sure if the scope of our project is narrow enough so as to ask 100 different people a similar set of questions since we’re currently exploring a number of different applications, the 15 minute interview format is promising for conducting surface-level research that we can use in narrowing down our verticals.

Customer discovery as a method clearly fits better with a product-out approach as opposed to a customer-in perspective that UX is focused on. So the use is clear in our case: go around asking people about a few pointed questions regarding their use of sensors, cameras, and data to try and find some real pain points.

We’ve also set of interview questions reserved for diving deep into building an understanding of a research participant’s life. These questions are reserved for interviews of participants from verticals that we’re confident we want to dive deep into. By utilizing two different interview methods, we’re remaining flexible with time while collecting data on a broader spectrum.

Next Up

Our next article is going to take a look at some of the insights we’ve generated through our first few interviews.