The research stack for better product decisions

Generate rapid insights for rapid growth

Being customer-driven has probably never been more difficult. Features are easily copied, markets quickly become saturated, bugs are publicly Tweeted about, and customer expectations are continuously increasing. Every time our team thinks we’ve finally achieved ‘product-market fit’, the target seems to move and our backlog grows. I know this is a familiar feeling for many founders and product leaders, and even more the case for those who haven’t gotten to market yet!

After writing about our rapid growth at Coderbyte, I got asked about how we maintain customer-obsession at our scale. As I mentioned in the article, we still know every customer on a first-name basis and to a great extent believe in Basecamp’s renowned ‘read all feature requests and then throw them out’ approach (based on the notion that important feature requests will keep “bubbling up” until you have to resolve them). But the reality is that we do operate with a roadmap, and it isn’t prioritized based purely on inbound feature requests or what’s top of mind for us.

There are a number of problems with exclusively relying on inbound feature requests to drive product development:

• The vast majority of our market opportunity consists of companies that aren’t our customers yet. Non-customers may have completely different feedback from current customers, which is why they’re not yet your customers. We can’t only optimize for the customers we have or we might marginalize future buyers.
• Many customers never give feedback unless they’re directly asked for it. Inbound requests tend to come from the loudest and most passionate customers, not necessarily the ones with the most common and addressable pain points.
• Feedback is incremental but product development shouldn’t always be linear. Feedback on something we’re currently offering will never help us discover if there’s a completely alternate and better approach that no one has ever thought about. Our customers are biased by the product we have today which by definition limits the boundaries of their reactions. In a future article, I’ll write about how we’ve done research and built ‘killer features’ that no one else in the industry had ever thought about.

While we do receive hundreds of unsolicited inbound feature requests per month that help to inform our roadmap, we also use a variety of great research tools to actively generate insight into potential product opportunities and gaps. I’ll explain briefly what each is and the top ways in which we use them.

Productboard — Feedback repository and idea prioritization

First and foremost, we store all product ideas in a simple kanban on Productboard. The killer feature they offer is the ability to forward ideas and feedback to an email inbox which automatically stores the email and enables us to ‘attach’ it to ideas as evidence.

Examples of how we use it:

• We add virtually every product idea to the backlog in Productboard, and are able to give each feature a rating across custom criteria like “churn reducing” and “enterprise-grade”.
• When we’re committing to building a feature, we can check all the feedback we’ve ever received about that feature in Productboard.

Typeform — Simple and engaging surveys for quick feedback from customers

We use Typeform because it’s engaging and interactive for participants, and because it has incredible embedding capabilities enabling us to seamlessly integrate it into the user experience.

Examples of how we use it:

• After a customer engages with a critical feature on our platform, we always have a link to a feedback form powered by Typeform. The responses go directly into a channel on our Slack.
• When we’re thinking about improving a feature or adding a new one, we add a ‘fake’ button for the feature that, once clicked, says: “This feature is still a work in progress. Click here to share your feedback.” The link goes to a Typeform asking what they thought the button would do so we can better understand expectations.

Session Rewind — Incredibly accurate and affordable user session videos

There’s a difference between what customers say and what they actually do. That’s why we have Session Rewind installed which captures every single mouse movement and click and lets us rewatch sessions of our customers on the platform. We use Session Rewind instead of FullStory and Inspectlet because it is significantly cheaper at scale, doesn’t have a bunch of confusing features that we don’t need, and seems to be a lot higher fidelity (sometimes certain clicks and interactions aren’t displayed in recordings generated by other tools).

Examples of how we use it (which I share in more detail here):

• If a customer tells us they had difficulty navigating a feature or experience, we query their session to watch it and see what they did. Over time we are able to improve usability by noticing patterns of customer confusion.
• When we release a new feature, we set a filter that Slacks us links to sessions where a user engages with that feature.
• We also have saved filters for initial onboarding experiences, cancelations, upgrades, and other key activities so that we can observe ‘aha’ moments.

Tetra Insights — Analyze and share user interviews

One-on-one customer interviews and conversations are so powerful and enlightening, but recordings can be really difficult to share with coworkers who don’t have the time to watch them in full. Tetra Insights generates transcripts and lets us slice and comment on individual videos, so our team can get the most out of each interview. We store each ‘snippet’ in an indexed and easily searchable repository.

Examples of how we use it:

• When we interview a customer and they say something truly eye-opening, instead of sharing the entire interview video (which no one has time to watch), we create a snippet of just the key statement and send it to the team to watch.
• We tag every topic mentioned in every snippet, so when we’re ready to address a pain point we can easily locate every time the topic was mentioned across all of our customer interviews.

G2— Honest testimonials from happy and unhappy customers

Sometimes it can be awkward for a customer to share their feedback directly, especially if it’s negative. G2 is a great channel for learning what your customers (and competitors’ customers) really think, especially because each review must include a section on what reviewers like and dislike about the product.

Examples of how we use it:

• We sort by 1 and 2 star reviews and analyze the biggest complaints. Are these customers with some very bespoke use case? Did they encounter a super rare bug? Or do they have a legitimate point about a product deficiency that we need to address?
• We run the same analysis for competitors but in reverse. We sort by 4 and 5 star reviews to see what reviewers love about our competitors’ products. This helps us address any blind spots or gaps in our own product.

Heap — Quantitative analysis of user journeys

While we bias toward qualitative and explainable insights, we do use quantitative data to understand the extent of effects. Heap is a click and page analytics tool that is both great at data visualization and unraveling individual user journeys.

Examples of how we use it:

• Visualize total adoption of new features over time.
• Create alerts when a certain conversion rate goes above or below a threshold. For example, if we redesign a page and inadvertently drop a button below the fold, we need to be notified if adoption plummets.
• Reverse engineer the “average” journey and flow that led to adoption of a new feature.

Those are the research tools that have helped us grow rapidly to 1,000 customers in less than two years. We’re always looking to gain an edge, so if there’s a tool you’d recommend, let me know in the comments.