Get out of the building: Why, when, and how to interview your customers

Published in

Uncork Capital

8 min readFeb 14, 2024

Image via Dall-E of one person interviewing another and taking notes

“Get out of the building.” Sounds like something you scream while watching a horror movie — but it’s also sound business advice.

As an undergrad learning about technology entrepreneurship at Stanford, two of my professors, entrepreneur Steve Blank and venture capitalist Ann Miura-Ko, told us to “get out of the building” so often that it became a mantra. Their point: when you’re building a new company, you need to talk to your potential and existing customers early and often.

Their advice has become a cornerstone of my career. Over the past decade I built and launched multiple products at Stripe, Twitter and Uber, and each one greatly benefitted from feedback gleaned from customer calls. At Stripe for example, my team led a program called “win/loss interviews,” in which we called everyone who had previously had at least two calls with a member of our sales team, regardless of whether or not they became a Stripe customer. Some of the sales we lost led to some of our most valuable insights about our product and sales process.

I’ve shared the guide I used at Stripe with a few of the founders I’ve worked with and since they’ve let me know that it’s been helpful in their own internal processes, I thought it might be useful to circulate it more broadly.

Why should I conduct win/loss interviews? What types of companies do they benefit?

We’ve all heard the YC adage: “Make something people want,” but how do you actually do that?

To start, you have to really understand your customer. Win/loss interviews are an easy, highly effective way to build in mechanisms of regular customer feedback that can pay dividends long term. They’re especially useful for growth-stage companies with multiple product lines that are starting to pour fuel on sales and marketing. But, the earlier you start incorporating systems for collecting and incorporating user feedback, the better. These types of interviews tend to be most effective for B2B companies, where surveying even a small handful of customers can provide meaningful feedback for product and go-to-market teams. They can also be a useful mechanism to align product teams with go-to-market teams.

What types of feedback can I obtain from these interviews?

As you’ll see in the sample interview guides I’ve shared below, these types of interviews are a great way to test the efficacy of your product and feature set, as well as your marketing channels, messages, and sales process. By conducting these interviews, you will be able to assess whether there is a delta between your perceived value proposition and competitive differentiation, and how your customers judge your strengths and weaknesses. You might think your absolute killer feature is X, but if your customers don’t value X, it doesn’t really matter. Similarly, feature Y might be one that was trivial to build, but it may be the primary reason your customer signed on the dotted line. These insights can help inform how you position your product moving forward.

By asking about the customer’s product research process, you can sometimes uncover valuable insights about where your customers seek inspiration for new products and services. As a fast-growing startup, especially in tricky economies, resources are often tight, and focusing your most valuable resources where it matters most is hugely impactful.

These interviews can also be helpful to uncover details about the buyer journey and the personas that are involved with the decision to use your particular product. For example, the Stripe sale often required buy-in from both engineering and finance. Sometimes, especially for early career AEs, the sales team doesn’t discover the implicit stakeholders behind the scenes in a particular sales process. Uncovering this with a non-sales interviewer can help refine your sales strategy and collateral moving forward.

One thing I particularly love about win/loss interviews is during the loss interview, when the sales team might have recorded the loss reason as “not the right time” or “lack of budget.” In these instances, by asking the questions creatively, you can often uncover the true reason the customer decided not to move forward. For example, maybe the customer didn’t fully appreciate the benefits you offer or couldn’t get the required internal buy-in to move forward. Sometimes, we’d discover that we lost a sale because the customer chose a competitor that they claimed had features we didn’t, when in reality we just didn’t advertise the fact that we offered similar functionality. While frustrating to discover after-the-fact, these types of “aha” moments can help prevent similar mistakes moving forward.

What, When, Who, How of win/loss interviews (when should I do these, and how do I fit them into my insanely busy schedule?)

At Stripe, we revised our filters for when we conducted win/loss interviews (e.g. minimum contract value, minimum # of sales contact points, specific product areas, etc.) over time. Note that the “lost” customers are often much harder to get on the phone, and the response rates will be lower, but they’re still very much worth trying for. We would send a simple email asking for 20 minutes of their time and offer to send them a bottle of wine as a thank you. You should make it as easy as possible for the customer to say yes to a call. Having a standard template (see below for mine) can make the calls very efficient.

I find win/loss calls are most valuable within about a month or two of the final sales decision so that the details and impressions will still be fresh. In terms of who conducts the interview, that can vary company to company, but my main piece of advice is that it should NOT be whoever manages the bulk of the sales process. It could be a product manager, product marketer, BizOps manager, or similar function.

After conducting the interviews, we would collate the feedback and share it with a broader team on a quarterly basis. At Stripe, this was typically a cross-functional team that included marketing, sales, product, and engineering. We played around with different formats, but sometimes would have every stakeholder pick one interview report to read in full, and then pull out highlights and lowlights on post-it notes and categorize them around the room. It was always pretty enlightening to see the piles of similar post-it notes emerge as we were able to collaboratively pull out the common themes of feedback across interviews. Now, there are tons of AI tools that can ingest the raw interview notes and pull out the most important themes and takeaways.

Maintaining a repository of key takeaways from win/loss interviews over time can be hugely helpful. Often, one or two individuals hold a nuanced understanding of customer perception and market feedback. Conducting, documenting, and distributing the takeaways from these interviews can be a valuable onboarding tool as your company scales.

There are countless ways to collect, evaluate, incorporate, document, and share feedback from customers. I hope this particular example of win/loss interviews is useful to you, and I would love to hear from you if you do decide to try this out!

Brief notes on the guides

Below, I’ve shared the interview guides my team used at Stripe. Not every question will be relevant for every customer, so you should customize these according to the customer and your company. Be mindful of time! If you’ve promised your customer you won’t take more than 20 minutes, stick to it. Don’t be afraid to skip questions if they feel irrelevant; the next question may uncover more useful insights.

Sample win/loss interview guides:

WON:

Intro

Thank you so much for taking the time to speak with me. I want to spend this time learning how you think about [Acme], how you found [Acme], and how [Acme] could improve. I’ll share all of this feedback with our teams to make sure we take action on it. Please be as candid as possible; this will help [Acme] improve.

[Acme] brand/awareness / consideration (10 minutes)

Great, now I’d like to talk about how you found [Acme].

Can you remember what was happening at your company when you started to evaluate solutions like [Acme]?
Were you using any products before [Acme]? Which products?
Are you still using [product name]? Why / why not?
What made you decide to look for another product?
Can you remember how you first learned about [Acme]? Tell me about that.
Were you evaluating any other products alongside [Acme]? What were they?
How would you describe [Acme] vs [competitors]?
How would you describe [Acme] to someone who had never heard of us?
As you were evaluating products, did you do much research? Tell me about that.
Where did you look? Why there? [Keep probing around the research.]
In your research process, what was most helpful for you in evaluating [Acme]?
How important were our website / docs / one pagers / guides when you were starting to evaluate us? Do you remember anything that was particularly helpful or information that we didn’t provide you wish we had sooner in the process?
What about [Acme] vs other products made you go with [Acme]? Why were these things important?
Can you remember who at your company made the decision?
Was anyone else involved? Who?

[Acme] strengths and weaknesses (10 minutes)

Now, let’s talk about your experience with [Acme] today.

Now that you’ve been using [Acme] for <x time>, how does it compare to your expectations? Why?
[Probe around strengths]
[Probe around weaknesses]
What works well? [time pending: Can you show me?]
What doesn’t work well? [time pending: Can you show me?]
What’s missing from [Acme] (if anything)? Why?
If you were to give advice to our Sales and Marketing teams on how to better reach customers like you, what would you say?
How frequently do you think of looking at competitors, if at all? What leads up to those moments?
[time pending] What are the characteristics of businesses who you think SHOULDN’T use [Acme]? Why not?

LOSS:

Intro

Thank you so much for taking the time to speak with me. I am hoping to use this time to learn from you how we can better serve customers from you, what we need to do to improve, etc. Do you still have 20 minutes to chat?

[Acme] approach (7 minutes)