Some arguments weren’t meant to be solved by you.

The Best Advice I’ve Received as a Product Manager

I asked a manager of mine once what he thought made for great software product managers, and he replied that the most reliable predictor of success he’d even seen was how much time they spent engaging with the market. Technical acumen, communication, and agile development know-how are important, but without constant eyes and ears on what your buyers and users need, little else matters. I agree with him, although I’d add a slight twist for newcomers to the product role today: know when to substitute market feedback for opinions, and make it happen.

One of the toughest things I’ve found about working in startups is that really smart people who want to move really fast can make really bad decisions. When people need to move quickly on a project, one of the ways they often cut corners is by substituting their own logic, experience, and opinions for real market validation. Products that are commercial failures often owe their lack of success to a misunderstanding or outright ignorance of how much the customer actually cares about it. You might think it takes a fool to skip that part of the process, but it’s easy to see how this happens. The pathology usually goes something like this:

  • A project gets underway without a solid understanding of the problem or how the solution creates value for the customer.
  • Under pressure to get started, the team moves forwards with the knowledge they have, making their best guesses about why the customer will care about the solution (I call these guesses “frog DNA”, after the gene sequencing technique from the movie Jurassic Park)
  • More decisions are layered on top of the untested assumptions at the project’s foundations, resulting in a brittle, untested solution
  • The product/feature/service bombs when it finally meets real users and testers whose needs don’t match with what the team assumed

When this sequence unfolds, the results usually speak for themselves. It doesn’t matter how strong a company’s brand is or how smart the executives, product managers, engineers, or consultants are. Whenever an organization relies on untested assumptions about whether a customer will value a product enough to use it or buy it, they crank up the risk of a project failing.

Ask the Simple Questions

The question is, why do we cut corners on market validation sometimes in spite of our better instincts? If we set aside the significant matter of not allocating enough time for research and due diligence, I think the next biggest reason is plain old hubris. We overestimate our ability to figure out what customers will pay for without our desks, which turns out to be a great way to make stupid, avoidable mistakes. I’m reminded of an Economist article from a few years ago that described how Adidas successfully retooled its brand image and product lines over the course of a decade working with an innovative market research firm. One thing the researchers did was to send disposable cameras to hundreds of customers asking them to photograph something that made them exercise. To their surprise, around eighty percent of the women sent back a picture of a little black dress. The company had long assumed that most customers were training to be good at specific sports, but in fact the overwhelming majority just wanted to look better.

Whoops.

Think about what a profound shift that implies for a business like Adidas. All of a sudden, they realize that all those global ad campaigns highlighting serious athletes scoring goals in intense training sessions resonated with at most 20% of their customers. Most of their product design decisions were also correspondingly wide of the mark. You might ask how an apparel company could miss something so fundamental about their customer base? But if you’ve ever built products you might see shades of your own past “whoops” moments. Picture being a junior employee surrounded by all of the marketing and creative veterans at Adidas and asking whether people actually care about being able to run past a defender or dunk a basketball. Imagine the glares that would come back! “Listen up, kid: we make elite products for elite athletes. Do you understand the technology that goes into making a running shoe this advanced? This is how the industry works.” And on and on.

I don’t know what it was like changing the mindset at Adidas, but I’ve experienced the process of releasing a product or a feature built on shaky assumptions. Sometimes I was the one asking the tough questions, and at times I’ve also been the one giving bad answers. Those disagreements can be tough, and tensions can flare when people perceive your questions as threats to their credibility. That’s why in these situations, outside expertise is vital to breaking the logjam.

Debating the Value of a Product or Feature? Settle It Outside

When you lock horns with a colleague on issues that are fundamental to a project or the business in general, it’s natural for people to sometimes dig their heels in to defend their ideas. The most intractable disagreements come from two or more people with the same experience and facts at their disposal arriving at opposite conclusions. When you combine the digging in of heels with the time crunch on a project, it’s clearer why questions like, “Why do people want to buy our running shoes?” are sometimes answered based on the influence and rank of opinion-holders rather than, you know, what customers say. The best thing you can do in this situation is bring outside information to blast through the opinions and get better answers. Fortunately, there are more tools available than you might realize.

There have never been more options available to someone who needs to quickly validate ideas with external feedback than today. Whenever you find yourself debating what your customers want or how they’ll respond to something new, save everyone on your team some time and aggravation by being the one to say, “Why don’t we test this?” There are lots of options out there, but I think of them under two broad headings:

Get Expert Advice — You know how the exact same argument sounds so more convincing when someone outside the building says it? That can be a great way to settle a dispute. Setting up a few quick conversations with outside experts on whatever you’re debating can be a great way to accelerate a decision. For under $1,000 you can have a handful of conversations with relevant experts who can help you reach a decision (you generally negotiate hourly rates with experts). I’ve had luck with Zintro for setting up phone screens with consultants on the cheap. There are other traditional expert networking services that you can use as well such as Gerson Lehrman Group which are pricier. And though it can be time-consuming, I’ve actually had the most consistent results cold-messaging people I found on LinkedIn with the relevant background and offering them $150 for an hour of their time. Consulting services have never been so cheap.

Survey the Market — To paraphrase Jeff Bezos, nothing ends an argument or flattens a hierarchy better than customer feedback. Even better, this doesn’t always need to be a lengthy, scientific study. Getting a few hundred data points (maybe fewer) is better than nothing at all. If you have a website that gets significant traffic, website polls can be a great way to get quick feedback on what the market thinks, and there are lots of free web polling widgets out there. If you want to talk to people who aren’t your customers yet, or if you’re targeting a more general population segment, Google Consumer Surveys lets you spin up a survey fast and collect results for anywhere from $0.10 to $3.00 per completed survey. And finally if you have a big email list that you can tap, you can always run surveys through SurveyMonkey, MailChimp, and lots of other tools.

The key thing about all of these options is that they are fast and low cost. If you’re stuck on an important decision and you sense the team falling victim to armchair logic, be the one who says, “prove it.”

A few suggested follow-ups for product managers: