Jyoti Bansal’s Secret to Managing a Startup’s Product Roadmap: The Four Lists
By Jyoti Bansal and John Vrionis
A crucial aspect of every successful startup’s journey is the ability to figure out what to prioritize when it comes to the product roadmap. As every founder knows, there are always more things to build than there are people to build them or time to deliver. At Unusual Ventures, we strive to help entrepreneurs succeed at the seed stage and a big part of that is helping them figure out what to build first and to deliver.
Recently, my partner Jyoti and I were working with a group of founders in Unusual Academy and we found ourselves digging into the process of product roadmap prioritization. I listened with admiration as I heard Jyoti explain in great detail how to manage the competing product requests that result from the different types of functional conversations — sales, customer success, engineering, and the CEO/CPO — a startup has, and ultimately deliver a prioritized product roadmap that the engineering team produces on a schedule everyone can count on. He calls this “the Four Lists” process.
We all see the world from where we sit. Ask a salesperson what should be prioritized in terms of the product roadmap, and you will hear about features that “must be delivered in order to close the key deals they are working on.” Ask customer success and it will be “functionality to make sure we don’t churn these existing customers.” Ask engineers and the list is quite often focused on the “technical debt” we have to make up. And finally, ask the CEO or CPO and the priority features typically revolve around what advanced functionality the company needs to deliver “to beat the competition and be a leader in the market.” Whose request do you prioritize? How to decide?
A big part of what we do at Unusual to help founders and management teams at startups realize their potential is to distill the wisdom of master practitioners. Well, here’s how Jyoti, one of the very best entrepreneurs of this generation, and his teams have learned to make product priority decisions at AppDynamics (acquired by Cisco for $3.7BN) and now at Harness (which is well on its way to being another unicorn) in his own words.
List #1: What Sales Wants
The objective of List #1 is to build functionality that enables sales to win new customers. Sales is in the field on a daily basis fighting hand-to-hand combat. To product leaders, the message from sales is usually “if we just had X we could close this six figure deal!” Every competitive situation could turn in our favor if we only had certain features. At Harness, we call these “opportunity gaps.” If you want to stay competitive, you have to close those gaps. Some features you may want to build quickly to compete in that same deal cycle. Others you want to prioritize and build over the next three to six months, so you can keep winning against your competitors over the long term.
When you’re building out this list, ask yourself, “What do you need to win in this market?”
At Harness, we put in place a system for every opportunity in our pipeline and use a tool called Vivun (one of Unusual’s portfolio companies) to track these opportunity gaps. That gives us a dashboard, where we can quantify how “expensive” each opportunity gap is for us. As an example, consider a scenario where one particular opportunity gap came up in five different opportunities in the pipeline and each of those opportunities is worth $1 million of pipeline. Tracking gives us visibility into which product features would be just “nice-to-have” vs. massive costly gaps and allows us to prioritize based on dollar impact. The second trick we always use is the win-loss analysis. Every time we lose a deal to a competitor, we ask the prospective customer for the technical reasons that we lost or whether there were other reasons.
Gaps from the field help drive decisions on multiple dimensions. Revenue is a big one, but these gaps also help prioritize based on severity of need (i.e. will it cause the prospect not to buy vs. just cause friction in the deal?), the segment of the need (i.e. are we meeting the demands of our target market or are we being pulled in new verticals, geographies, company sizes?), and the competitive impact of the need (i.e. should we bolster areas against our top competition?). For example, your sales list will start to look different if you prioritize for moving into the enterprise, expanding internationally, or taking out a competitor.
Keeping track of product requests with List #1 from Sales is incredibly important. But that’s only one consideration — or list — of product features you (as the CTO, CEO, Founder, or Product Leader) should be thinking about.
List #2: Existing Customers
As product leaders, acquiring new customers and tracking the opportunity gaps are unfortunately not the only features that require consideration. If you focus only on closing gaps with competitors, you would neglect your existing customers and what they might need to continue to be successful. List #2 focuses on those users who already use your product and what’s important to them. Often times, existing customers discover they need certain features that they didn’t think of at all in the evaluation process. That’s because as they implement the product and go from being new users to power users over time, they start finding gaps in the product and begin asking for different capabilities. If you don’t listen and demonstrate an ability to offer those features that they need, those customers become dissatisfied and are less likely to renew. One example might be a customer that needs an integration that works with their internal user database, so they can pull in all their users. If you don’t build that feature, their adoption of your product would be limited to a small number of users and they would find decreasing value in your product over time.
The most important thing to consider for this list is being proactive and looking at customer usage and adoption as key metrics. That means examining the data on user engagement and seeing if a customer is becoming increasingly disengaged. You don’t want to wait until they churn, and ask them after the fact why they stopped using your product because by then, it’s too late. Instead, you want to collect product requests from existing customers before they churn and really seek to understand the reasons why customers are not using the product as much as they once did.
At Harness, for every account, one point person is responsible for putting together a list of three things that that customer needs to succeed and three things the customer needs to succeed even more. There are a number of tools that can help you track this. At Harness, we use Natero to keep track of these requests and score customers red, yellow, and green. (Red meaning they’re using our product very little, yellow meaning they’re using the product as expected, and green meaning they’re power users of the product.) We use this scoring system as the leading indicator to determine if our customers are happy and whether or not they’re likely to renew.
Once you have this information about customer satisfaction and requests, there may be several reference points you use to determine which product requests to take on. For instance, you might consider the ARR value of the account or whether the customer is in a strategic vertical you want to dominate. Or maybe it’s a small account, but the customer is really upset. (We refer to this as reading the “temperature” of the customer.) All these things might factor into your prioritization for new product features, but it’s hard to be formulaic about it. At the end of the day, you have to make a judgment call as product leaders.
Delighting customers is extremely important for any startup these days. Almost every successful company has really high metrics for customer retention. So that’s really the second list you have to start tracking and work towards.
List #3: Technical Debt
List #3 is maybe less exciting, but just as important as the other two lists we’ve described. As you quickly iterate and scale as a startup, you start accumulating what people call “ technical debt”, which you chose to take on as a tradeoff to build your product quickly. That technical debt may require you to re-architect a system at some point or slow down and work on test automation. These are not customer-facing features so they don’t impact the sales or customers, but cleaning up technical debt is something you have to keep doing to maintain and/or improve product quality.
List #3 is where your engineers track any accumulating technical debt. This may mean pausing or slowing down while you make improvements. Other times, it may mean redesigning a certain feature or writing more unit tests. Working through technical debt can also mean cleaning up bad code that has been assembled for some time. In terms of how you incorporate this into your sprints, companies dedicate every 3rd or 4th sprint to technical debt. Others prioritize a couple of items from this list in each sprint or allocate a fixed bandwidth in each sprint to technical debt. In any case, don’t go too long without working through your technical debt. If you don’t take care of your technical debt and keep building more features, eventually your product architecture gets so unwieldy that product quality suffers and it creates more serious problems in the long run. That’s why it’s critical to maintain this list and track it as you build out your product.
List #4: Innovations
The fourth and final list comes down to engineering toward future growth. It’s about building innovations that your customers are not yet asking for, but are designed to expand your total addressable market. In Unusual Academy, we teach our founders to find their beachhead market and expand from there. That means constantly thinking about your next innovation. Ask yourself, “What do I need to do to expand my total addressable market?” You can’t depend on existing customers to source these innovations because they often aren’t the ideal customers for new features or product lines. They care about what they bought, not about what you don’t have yet.
For example, our first product at AppDynamics was Application Performance Monitoring (APM) for Java applications and our customers who were Java programmers were asking for certain features. That was our beachhead market and we had to deliver against Lists 1 and 2 to stay competitive and delight customers in that space. But at the same time, we had a playbook for expanding into .NET applications and had to build out this capability, which most of those existing customers were not requesting as they didn’t have any .NET applications. Eventually, we built out our third product, End User Monitoring (EUM) and our fourth product, Synthetic Monitoring as we continued innovating and expanding our market.
Essentially, you have to approach this fourth list by working backwards from your revenue goals. Say you want your revenue to be $100 million ARR five years from now, $30 million ARR three years from now, etc. You have to have enough addressable market for your product, so you have to ask yourself what you need to do to keep expanding that market and hit those revenue numbers. That means you have to keep working on product expansion and innovations. Otherwise, your growth will slow down.
Balancing the Four Lists and Making Tradeoffs
Ultimately, whatever you decide to build comes down to those four buckets, which is what any product leader should be weighing when they make tradeoffs and define the priorities for the product roadmap. Some requests on the lists are lighter lift and can be done immediately, like a bug that needs to be fixed. Others take a little more effort and can be accomplished in the medium-term, like slight product enhancements. Then there are the major product requests, which require building out completely new services and features for new customer adoption. There’s variation in the magnitude of requests that the product leader has to balance.
Whether you’re the Head of Engineering, Product Manager, CTO, CPO, or startup CEO (and wearing all those hats at the same time), tracking these four lists can help your team align and ensure they are focusing on the right things. But how exactly do you get the balance right between these four lists? It’s a very tricky balance to pull off and you will have to adjust the balance as business priorities change. That means that at any point, you may want to spend more time on one list over the others. Say you’re losing a lot of leads to competitors and are struggling to get revenue going. That means you probably need to put a lot of your focus on List #1 and make the conscious, deliberate decision to slow down on technical debt or innovations and market expansion.
Or maybe you’ve sold a lot of product, but your customers are unhappy or you are having quality issues and churning customers because you’re not keeping up with the things that customers are asking for. Then you may want to slow down on the other lists and spend your energy on List #2 and List #3. In an ideal world, you want all four lists to be executed extremely well at the same time. But in the world of startups — where time and resources are constrained — finding the right balance right is key.
Structuring Planning Cycles
One thing we’ve found helpful at Harness is putting in place a system where every three months, our team goes through what we want to achieve and creates a planning cycle based on that three-month timeframe. Then, the team does an additional mini planning cycle every two weeks. It might not perfectly match up with the three month planning cycle, but it should align more or less with the broader three-month goals. Finally, we look at what we ship every day. By tracking those daily, two-week, and three-month sprints, we can see how we are making progress across the four lists.
Once a week, our CTO Rishi will meet with our Sales Engineering team to go over the first list and what is important to buyers. Then, he meets weekly with the Customer Success team to understand the second list and what’s important to our existing customers. He also meets weekly with engineering managers and asks for their lists of technical debt. Once he’s accumulated the lists from key stakeholders, he and I talk about the product expansion and innovation list every few months. Then, he balances what goes into a two-week sprint. It’s impossible to implement all ideas across all four lists all at once. Instead, you want to make sure that you are systematically going and adding the next 10% of your vision and going from 30% to 40% and 40% to 50% and 50% to 60% and so on until you reach your full product vision.
Unfortunately, unlike like a social product where millions of users can give you instant feedback, you don’t get immediate feedback on the enterprise side. It may take you months to get the data points to understand if one of your new product features is effective or not. That’s why there’s no exact science behind this process and why it becomes even more important to use the four lists to track high-level trends. If you’re systematic about capturing which requests recur most often across each list, you can get a sense pretty quickly for the most prominent high-impact product features, allowing you to prioritize your engineering and product roadmap.
Using the Four Lists process, you might wonder how the feedback loop works to update the company on the status of what product features were picked and how they were chosen. Sharing this framework/concept with everyone at the company gives everyone the important context that there are multiple requests to balance. At Harness, we have one wiki page in Confluence where everyone can see the lists and we mark progress in Salesforce using Vivun, so the sales team knows which new product features we’re working on and when each will be ready to roll out to customers.
Another thing to keep in mind is that as your company grows, this process changes as well. Usually, you start as a “single product” company and stay that way for the first few years or so. Once you have multiple products, you will likely need to have separate Four Lists for each of your products.
Many Ways to Succeed
At Unusual Ventures we recognize there are alternative methods to solving this challenge of product prioritization. We wanted to share this concept of “The Four Lists” approach because we’ve seen it work and we hope we can help every founder in some way. Enjoy the journey and good luck! Onward!
Interested in learning more about Unusual Ventures? Follow us on Twitter.