Roadmap Gut Check: Questions to Ask Yourself Before Building Anything

Product managers (PMs) often make the same mistakes when they start their careers. I’m sensitive to this, because I’ve made the same mistakes myself. To help new PM hires at Cockroach Labs get better faster, I give them four questions to ask themselves to help determine if they should add features to their roadmaps. The questions are deceptively simple, but real-world pressures and biases make them surprisingly difficult to answer.

1. Do I Understand The User’s Problem?

PMs should deeply understand the core user problem they solve with every feature at a first-principles level. Many new PMs know this intellectually, but it’s one of the first steps they skip when starting their careers. I’ve seen this happen for a few reasons: sometimes it’s pressure from a sales team, sometimes it’s an idea from a passionate CEO, and other times it’s just an eagerness to get early wins and show value quickly. No matter the reason, it can be hard for new PMs to take a moment to breathe and understand what user pain point they are solving. Note: This question assumes you and your team have a basic understanding of who your users are; if you don’t, search the internet for “user discovery process” to get started.

Here’s a real-world example from early in my career at Percolate. I was managing a product for centralizing and indexing a brand’s digital assets, and a large company’s IT team wanted a feature to download files of different sizes and formats before it would move to our platform. As I probed why their team needed the feature, the customer grew more and more annoyed. The feature request was simple; why did it need clarification? How could I be the product manager for this product if I didn’t know something like this? I felt myself losing credibility as our sales and marketing teams listened in.

At this point, I felt a huge pressure to just take the info I had and run with it; I could surely build what they wanted. However, I knew, deep down, that I didn’t truly get why that customer needed that feature. I decided to power through. After a few more painful minutes, I finally got to the root need; the raw uploaded photography was too big for social networks to publish, so they needed smaller photos to publish through our platform.

Armed with this information, my team was able to prototype, and ultimately build, a much better solution. Rather than use a file size selector, we could connect the publishing system to the asset manager and automatically infer the exact size and format requirements from a social network and do the conversion behind the scenes. This meant users could just pick the photos or videos they wanted and publish them, without worrying about any other manual steps in between. We eventually won that customer’s business and were able to provide a great asset management experience for all our content marketing customers.

The path to really understanding the “why” may be awkward, but it’s worth it to spend the extra time to get it right.

2. Does this support my company strategy?

In a nutshell, a product strategy outlines the activities you take to build stuff that people love and is hard to copy. Before you build anything, take the time to make sure it makes sense, given your company and team strategy. If you don’t know what your strategy is, that’s fine. Just take a step back and align with your manager. If your manager doesn’t know, there are good books (here’s one!) for helping your team arrive at one.

Once you have a strategy in place, use that to quickly de-prioritize feature requests. If you don’t, you’ll be stretched too thin and will lose to more-focused competitors. PMs of all skill levels make this mistake — they underestimate what it takes to build in multiple directions. This comes up a lot in B2B product management: “What’s one tactical feature here or there, if we can win a nice new customer logo?” New features in new directions come with new bugs and new enhancement requests, unfortunately. Even more alarming in the B2B example above; they also come with a new customer that specifically paid you to build what they needed, and that customer’s retention is sensitive to your ability to continue to deliver in that new, off-strategy area. Try your hardest to avoid this situation.

Here’s an example of how I used strategy to simplify prioritization at Cockroach Labs. The mission at Cockroach Labs is to “make data easy,” which provides a broad North Star, because there are lots and lots of activities that make data difficult: Analyzing large amounts of data is hard (which is why analytics startups continue to pop up on what seems to be a daily basis), but as databases move to the cloud, it turns out that simply storing data and getting it back quickly and reliably can be even harder. CockroachDB’s architecture is well suited for solving both issues, and when I joined as Head of Product, the roadmap already had features in both areas.

One of the first things I did was narrow the scope of our strategy to take Big Data analytics completely off the table in the short term so we could focus on online transaction processing (OLTP); OLTP is a much, much bigger market and we are more differentiated from our competitors here. Conceptually, making that cut was the easy part; the hard part was navigating the internal politics of cutting exciting features from the roadmap and insisting that our sales team push back on large, analytics-focused prospects (adding friction to the sales process was doubly difficult, because at this point, CockroachDB was pre-revenue). Internal education and communication highlighted our strategy, market opportunity and the competitive advantage we were building, helping make this transition possible.

Enforcing strategy during roadmapping is uncomfortable, and it’s something I didn’t do early in my PM career, but it ultimately helped CockroachDB get to product-market fit faster. For example, this additional focus let us take the time we had earmarked for analytics infrastructure and invest it in valuable areas where other databases couldn’t compete. This additional bandwidth for making progress on transaction processing let us deliver 10x the scale of Amazon Aurora at a fraction of the cost and, most importantly, land our first paying customers.

Use your roadmap to enforce focus, and when you truly have a hit (congrats!), you can broaden your strategy and the types of features you create.

3. Is This Worth Doing?

The process for determining the return on investment (ROI) of an effort is well researched, with net present value on one end of the spectrum and 2x2 cost-benefit charts on the other. In practice, though, gauging ROI is a complex, iterative process that involves scoping, experimenting, and uncovering the hidden costs and risks of development. Often, PMs forget that there are two components to ROI: return and investment. This is another thing new PMs know academically, but it falls by the wayside when they are juggling multiple features or working under pressure from external stakeholders, including customers and executives.

When I was a senior product manager, I was in charge of building a platform to take huge amounts of data from social media services, analyze it using natural language processing, and index it so brands could track what their customers were saying, how they were talking about the company vs their competitors and other metrics, including sentiment. I knew from a simple back-of-the-napkin calculation that this would be the most expensive product we ever built. Even so, I spent weeks with vendors to create models that would tell you the precise spend, how that would change depending on which features we built, and what we’d have to charge to break even.

Unfortunately (and embarrassingly obvious it hindsight), I spent all my time trying to understand the investment but didn’t put in the time to gauge the expected return. Would we be able to attract enough customers to make this worthwhile? And if not, how could we lower the cost of this system for it to make business sense? Ultimately, I built the product without having those answers, and management killed it a year later. I could have avoided this blunder, manually producing quick, one-off brand reports for customers just by paying for the small amount of data and processing services needed to produce them. If the demand for those reports was large enough, then I could make the case for investing in the much more expensive infrastructure to automate it.

The takeaway here is to always think through both sides of the return vs investment question and constantly think about how you can use prototypes and tests to evaluate each side as quickly and cheaply as possible. The book Inspired provides a variety of useful tools for identifying and reducing risk.

4. Am I OK with what I’m giving up to build this?

This last question is simple and powerful, but the catch is that answering it correctly gets harder and harder as your team and backlog grow. To know what features (or experiments) will fall off the roadmap if something new comes in means you are able to account for what’s happening in a given time, relative priorities, dependencies, costs, and — most importantly — what features are just on the margin of being in and out of scope. Answering this question consistently is less about your skill as a PM and more about your ability to account for the work in your backlog and keep your scope line up to date as you learn new information. I spend a lot of my time as Head of Product thinking about how my teams can answer this at scale.

If you add features to your release without thinking about this question, one of the following things will usually happen:

• A feature will slip. If you don’t have any promises or anyone who depends on what you deliver when, this may not matter to you. However, in larger companies or B2B companies that need some level of predictability to coordinate with sales, marketing, customer success, etc., this can be a major issue.
• No features slip, but you don’t build them as planned. In practice, this means cutting usability, certain workflows, or stability. This can pose significant risks to your customers’ confidence in you.
• The team will work longer hours. Unplanned or improperly accounted for roadmap additions can result in late nights, weekend coding sessions, and a burned-out team.

I have to say, early in my career this last one was tempting. If you can motivate your team to do more in the same amount of time, why not have it all? In fact, many of my early promotions were from delivering higher-quality features faster than other PMs. Sometimes my team and I spent all-nighters to ship on time. Pushing your team harder is a double-edged sword, though; it may be fine for a sprint or two, but in the long term that may be the worst way to go, as the human cost of engineering burnout can be very difficult to bear.

As a PM, your ability to push back is one of your most important skills; in fact, you should consider it a requirement of your job. Keeping your roadmap or backlog up to date so you always know what’s in scope, what’s on the bubble, and then using that information to talk in terms of tradeoffs on the margin is a great way to master the roadmapping process and quickly raise your skill level.

—

Think of the questions above as a quick gut check during your roadmapping and discovery activities. If you can’t answer all of them today for everything you’re building now, that’s OK. Just pushing yourself to improve your ability to answer these questions will make a big difference as you grow as a PM. Good product decisions compound over time, and this approach will help you make more of them.