How User Interviews Changed the Direction of Our Company
How to conduct these conversations to inform your roadmap
“Talk to your customer.” Whether you’re an aspiring entrepreneur, a full fledged startup founder, a product manager at a big company, or anywhere in between, you know the importance of customer feedback when it comes to building your product. It’s by no means a revolutionary practice, and could even seem so obvious as to not be worth mentioning. If your customers say the pizza’s too salty, you use less salt to keep your customers coming back ¯\_(ツ)_/¯. A seemingly “duh” concept, adapting your company in the face of user feedback is all too often overlooked. When a team is deep in true build mode, spending every waking moment piecing together the million necessary components of a company to ensure it lives to see another day, the seemingly obvious things start to slip through the cracks. User feedback can sound like noise when you’re busy, but in fact it’s always valuable and should always be taken into account. Understanding our user feedback led us at Resultid to pivot in a way that defined our purpose and helped us to be more resilient, more useful, and ultimately more successful.
Time For A Pivot
After spending 2021 creating a Minimal Viable Product (MVP), we came to realize that we were solving big problems for a small market, and, perhaps more importantly, comments and usage from our previous customers suggested they valued parts of our solutions that we didn’t expect (like easy data exports or auto-generated reports). It became clear that the best direction forward was a reshaping of our vision and product altogether, though what this exactly meant was still hazy. We found ourselves spending a lot of time discussing big plans, white-boarding countless ideas, and bringing on new team members. As our company grew, we realized that while the previous MVP laid the groundwork for our amazing new team and the future of our product, we were still missing one obvious, not so obvious element: talking to customers to validate our new approach.
In hindsight, many of the early struggles we faced at the start of 2022 stemmed from a lack of external feedback. We are lucky though to have a stacked advisory team (think the 95’ Chicago Bulls). After talking to them about all our big, exciting plans for where we wanted to go next, we swiftly received a painful, but much appreciated, kick in the rear end. They knew that while we had a lot of the pieces to move forward to the next chapter of Resultid, we were missing the tangible feedback from potential users to validate everything we were saying.
Alerted to our shortcomings, we were energized and ready to go out and talk to any and everybody to hear out new users in different businesses and verticals we did not entirely understand…yet.
Sourcing User Interviews
Seeking direction from the needs of our users, we had conversations with as many of them as possible. Anchoring ourselves around a real milestone, we said that we wanted to have 150 interviews in 90 days. With this in mind, the first area to address was sourcing conversations with potential target users. Initially we went to LinkedIn, where we used tools like Expandi to programmatically maintain connection requests and messages to individuals asking them for a brief interview, but this was clunky and fell short. The conversion rate was lower than we had anticipated, and reaching our target of 150 interviews seemed like a stretch. After some more research, we came across Respondent and User Interviews, two platforms that connect researchers to participants quickly and affordably. (If you haven’t used these platforms, I highly recommend checking them out — you simply post a description of the research along with a survey to qualify participants, set parameters around the conversation (video on, recorded session, etc), and you’re off to the races.) Engagement rates soared and a sea of useful, qualitative data was on our horizon.
Ground Rules For Our User Interviews
After establishing a pipeline of participants, we needed to think about how to facilitate the user interviews. We wanted people to speak candidly and to have the freedom to elaborate, which is to say we were gearing up to gather a whole lot of complexly textured, qualitative data. The ground rules for every call would include at least two members of our team: one to lead the interview, and the other to take notes (more on that in a moment). This proved to be a great way to focus on the participant, address conversation topics, and ensure everything was captured cleanly. By recording our sessions, the transcripts could be revisited later if needed (after confirming with the participant that they were okay with this). On the call, the interview lead would do 99% of the talking, while the notetaker had a form up that required participant info (name, email, demographics, etc) and typed responses to each question (the conversation lead would tactfully follow the forms questions so the notetaker could follow along easily). We set each session to be 15 minutes long, and while I was dubious about the length of these conversations upfront, it turned out we could build rapport and get all the information we needed within that time limitation (just keep the session focused and on topic).
Questions and Data Capture
With the structure of our interview finalized, we set out to formalize the questions to ask and how to best capture feedback. Ultimately, we decided the best way to manage our conversations was to create a Google Form for our notetaker to fill out with all the questions we had for our participants and their responses. After the interview, the notetaker submitted the form, which was linked to a Google Sheet that allowed us to easily view each participant row by row and see their responses in bulk. On constructing the questions themselves, we focused on going broad, asking things like “Tell us about the hardest part of your day” and “What tasks are most challenging in your role.” Obviously, you will need to tailor the questions to your specific business case, but the guiding principle is to ensure you are never asking leading questions, or questions that will encourage a desired answer. Ask questions targeted towards what you want to know, without limiting or skewing your research by angling your subject towards a narrow answer.
Milestones and Dog Food
Within 32 days we achieved our goal of 150 calls, and at 90 days we had added dozens more, leaving us at 215 total; 65 more than we originally set out to achieve. This was 215 opportunities to talk with a wide variety of people in interesting and complex business roles, to better understand the difficulties they faced in their job, and how our solution might help them. Funnily enough, by gathering so many responses we created a problem that our very app was built to solve: qualitative data overload. With 215 calls of approximately 22 questions each, we had generated over 4500 cells of information in our Google Sheet, with each cell having up to a paragraph of written notes. That was a lot to look through. A lot. So, we tightened our laces, put on an extra pot of coffee, and got to work developing a solution that would allow us to easily upload this data and extract the themes from our own data set, rather than burying ourselves for weeks reading our language based, sentiment filled data. Reappropriating NLP technology we’d already been building, we developed a solution that some of our participants had described wanting for themselves (a shameless plug indeed, but what’s our tool worth if we don’t share it? If you are curious to check this out for your research data, you can try Resultid free here!)
A Journey Forward
Out of this process we learned the many ways analysts, product managers, ecommerce store owners, consultants, marketing managers, and much more, look at qualitative data and the challenges they face in their roles. We developed a scalable process that was able to generate meaningful insights from potential users to validate our own big ideas. This even led to the initial developments of our new product. What happened after was a series of events that led to an incredible summer and over 1000 users signing up to join our Beta. My hope is that there is a small bit of information in here that can help you on your journey to building an incredible product, and above all, I want to remind you…talk to the customer.
