Killing Your Darlings
Typically, people think of product managers at innovators, creators. We turn ideas into reality. But the secret shared by many PMs is that few moments in the job are more satisfying than killing features. We know that every feature is another bug queue, another user cohort, another training session for sales, another help doc to write. Another mouth to feed.
Sometimes, it’s glaringly obvious when a feature isn’t pulling its weight. It carries a lot of technical complexity. It’s not getting traction with users. It’s not helping customers convert through the funnel. No one’s going to shed any tears when that feature is put out to pasture.
But what about the other 90% of the time? As Oliver Young, PM at Twitter said, “Killing features is the hardest thing a PM has to do.” The decision to sunset a feature or product line is rarely quite so black and white. What do you do about the features used and loved by a small subset of customers?
To be, or not to be. That is the question that often comes up in times of transition or change. Maybe a new PM joins the team, or the product is undergoing a big UI overhaul. Once you know you are at the point where you need to make the call, there are some tools that can help confirm your assumptions and rally various stakeholders behind the decision to kill the feature.
Prove It: Qualitative and Quantitative Tools
Hide and kill. Juliette Senesi from Skyhook advocated a “hide and kill” strategy. Hide the feature or remove access for users with no external communication, and observe the impact. How many support tickets come in? How many angry emails? How many executive appeals? If the reaction is low or manageable, you’ve got the data you need to make the case for permanent removal.
Counting support tickets is directionally helpful, but you should also see which users are responsible for the increased volume. As Greg Sham from Fiksu put it, “How much should you cater to the vocal 1% of your user base?” Your loudest customers tend not to be your largest customers. Distinguishing between any user feedback and core user feedback is critical.
A/B testing. Make the feature available for one cohort of users and not another, then review their usage over a period of time. This may sound like a clean cut experiment, but if wrongly administered can cloud the issue. Users that are not prompted to use the feature will use it less, and if you draw attention to it, you’ll muddy the results. Be sure to review the data created from this exercise cautiously.
Identify the latent need. What is this feature really trying to solve? Are you solving a problem your users never really had to begin with? Or perhaps you’ve got a square-peg-in-a-round-hole scenario on your hands. Get out there and talk to your users. As Howard Kaplan, VP of Product at Queralt reminds us, the investment in spending more time with users always pays off.
Evaluate the actual cost of maintaining a feature. Product teams often underestimate the carrying cost of a feature. They think, “We invented this. Therefore it should exist.” What does it really cost to keep a feature around? How much additional sales training does it require? How much marketing spend is being allocated to it? How many support tickets come in per month? How many bugs does it generate? How many additional servers does it need? What’s the impact to customer load times? Once you’ve got a sense of the carrying cost (financial and organizational), you can have a more objective conversation about the future of the feature.
In product, as in writing, you must kill all your darlings.
Have frameworks to analyze the health of features. Product managers have an endless barrage of decisions to make, so it’s great to have a process to fall back on. When you build a new feature, set some OKRs or SMART goals upfront. If the feature doesn’t hit its numbers in the evaluation period, it’s time to have the conversation.
This is easier said than done. Data isn’t always a silver bullet, because people have a funny way of interpreting information that contradicts their gut. As Ben Horowitz mentions in The Hard Thing About Things, people only take action on positive leading indicators and only look for alternative explanations on negative leading indicators.
Howard Kaplan has used RFM (recency, frequency, monetary) scores as described in Jim Novo’s book Drilling Down to make the quantitative data less susceptible to interpretation. RFM can be used to identify your best customers on a quantitative basis.
You Made the Call: Dealing with Stakeholders
Let’s be clear: customers and sales teams will never thank you for killing a feature. It’s your job to consider long-term implications to your users and your platform. The dividend on sunsetting a feature comes from lower technical overhead, fewer customer support tickets, and improved focus. So how do you break the news?
Foster good customer communication: tell your customer success and service teams, and work with them to communicate the change to customers. Depending on the impact of the decision, it may require proactive outreach to all customers, or just a subset of power users. The key is to get ahead of the issue before it becomes a problem. Don’t let customers feel like you pulled the rug out from under them.
Prepare the sales and marketing teams: After customers, sales tends to put up the biggest fight when there’s talk of killing features. They hate when you take an arrow out of their quiver that could be used to close a deal. Expect pushback. They’re going to tell you about the contract they just sold based on that feature. They’re going to talk about how well it demos. The blog post that sings its praises. They’re going to try and sell you on keeping the feature. Hear them out, but don’t forget what led you to the decision in the first place.
Sometimes you just have to stand there and get yelled at.
Megan Conlin from OwnerIQ mentions that internal training is critically important whenever features are added or removed. Get sales comfortable with the new pitch before the change, and you’ll get them on your side.
Have a reason. If you’re taking something away from users, they’re going to want to know why, and you should have a good answer. It can’t just be low usage because inevitably, one of those users will say, “But I was using it!” You can blame high carrying costs, but users may not care very much about your cost structure. Juliette Senesi suggests referencing the evolving nature of product/market fit. Perhaps a feature that made sense for the market 3 years ago is no longer a good idea today. Now it’s about more than the user.
The lollipop approach. Whenever you’ve got bad news to deliver, it helps to have some good news, or an alternative solution to offer. The promise of something new and exciting can take away the pain of losing something familiar. It doesn’t have to be a brand new feature. It could be an explanation of how to use existing features to address the need intended for the old feature. Check out this great example from Bitly when they sunset their Bundles feature, or read how Google transitioned Picasa users to Google Photos.
You Identified It, You Proved It, Now Kill It
Once you’ve made the decision to sunset a feature, it’s best to just rip the Band-Aid off. Get the word out to the team, and set a date. Pick a length of time that’s appropriate for the customer usage cycle. For example, if you’re sunsetting a campaigns feature in an ad platform and the average campaign lasts 2 weeks, your sunset date should be 3 weeks. Your well-armed sales and customer management teams can handle communicating to prospects and customers as needed.
Making it Count
After it’s all said and done, you should take the time for a retrospective. Don’t agonize over whether you made the right call to keep/kill a feature. Look at how the decision was made and communicated, and see how to be better next time.
Shipping new features is a means to a greater end: delivering a great customer experience that contributes value to your business. Sometimes, you get more value from less product.