Confessions of a former Feature Factory PM

Forgive me Marty Cagan, for I have sinned.

I’m ashamed to admit it, but at one time in my early career, I was responsible for managing a product like a feature factory.

Not sure what that is? A feature factory mindlessly cranks out features like a production line, hoping the next one will change their fortunes.

And I did it. A lot. But not anymore.

In the attempt to redeem myself, I thought I’d share a few frameworks that can help you, your team or someone you know beat the trap of becoming a Feature Factory PM.

Outcomes over Outputs

Feature Factory PMs fall into one of two camps:

• No measurement
• Measure the wrong thing

The worst culprits of the first camp, that I’ve experienced, are small B2B companies adding features to win sales contracts. And oh boy, is that a bad place to be.

Features become a sales strategy. Success is measured in winning contracts from those features. Your product becomes a bloated mess. And for you, it can create a very non intentional mindset to product management. You might start saying things like, “Some things you just can’t measure.”

Bullshit. You’re better than that. Get yourself over to camp two — but pack light. You won’t be staying long.

Camp two is usually disconnected from the business, either because they’re measuring non-business metrics like story points and velocity, or they simply aren’t aligned to business strategy.

They might measure…

• Feature metrics: Performance of a feature (How many people are using it? Is it quicker or easier for a customer to do X now than it was before?)
• Product metrics: Performance of specific product objectives (Because of feature X, more people are using functionality Y)
• Team metrics : How many story points/features were launched at the end of the month/quarter? (I put this in for dramatic effect; this will only lead to — at best — success theatre, and at worse, the wrong behaviours from all parties)

This might seem like a very logical way to measure, but in reality these measures tell us very little about the business impact of all the work we’ve done.

A more business centric way to build and manage a product is by measuring our efforts in terms of outcomes, not output.

When we measure outcome, we measure the impact the new features had on the activity of the users AND the success of the company. When we measure output, we use features; when we measure outcome, we use OKRs.

If you haven’t heard of Google’s OKR framework I highly recommend you use it. It will give you a way to clearly state the what you are trying to achieve and how you will measure it. There are a multitude of great resources out to get you started, but I rather like Radical Focus by Christina Wodtke

But Hold on though, we already use OKRs.

You may already use this framework, but are you using it right?

Perhaps you do measure those product metrics via OKRs but are you measuring the business impact?

Here’s a fictional example below.

The team are working on a sexy existing functionality called feature X, which is a secondary feature in this new product. As it’s a new feature, they want to make sure it’s being used, so they are measuring product metrics and have an OKR set against it.

However, the business’s real objective is to increase their activation ratio. This is key — once activated users stick around with the product, retention is good.

Now our fictitious team could nail their product metrics, but could they succeed? If their metrics aren’t measured against the business objective, how would they know? What if adding just one of the features hit the business goals and the rest were diminishing returns? Without adherence to (and observation of) the business objective, we run the risk of building expensive features without return.

Make sure the OKRs align to business objective.

Now, there is a caveat to this. If you work in a large product company, your team might be many levels removed from the business objective. Usually this is achieved with cascading OKRs. Make sure you and your team are aware of the business objective and are still measuring it.

Discovering the right things to build

Feature Factory PMs tend to focus on the pace of feature delivery. They pride themselves on producing many features, without acknowledgment of the metrics they should be moving. They feel like it’s their responsibility to know everything, or least give the impression they do, but rarely invest time to explore the problem space. They just keep on building; because to them (or their company) it’s a bigger sin to not have developers building things than it is to have them discovering the right things to build.

Using a discovery framework like the Product Kata by Melissa Perri, will help you structure your approach to achieving the business objective, avoid poor ROI and better serve your customers.

How does it work?

• Understand where you are (the current condition),
• where you want to be (target condition),
• whats stopping you from making progress towards it (obstacle),
• whats your next step,
• what do think will happen (expected),
• what did you learn.

Here’s an example:

Consider this as the current condition — 85% of trial users don’t complete key activation steps.

We don’t know why yet, but we do know that we need to achieve a rate of at least 50% to have a healthy trial to subscribing rate and then retention rate thereafter.

It is here the Kata gives us a simple framework to progress through what we don’t know to discover potential solutions, then to communicate the findings and progress.

In this example below, each row might be a week or shorter depending on the problem space.

Example Product Kata after several weeks

The process is about progressively solving problems, identifying the next step, and expanding understanding. Once there is sufficient understanding to anchor knowledge and take the next step, do so. Step and repeat. This framework will deepen your customer knowledge and expose a host of opportunities and solutions to impact the business objective.

Use a discovery framework to establish and communicate what we need to know to achieve the business objective. Be clear about what we don’t know and what we’ve learned.

Opportunity solution trees or many eggs and multiple baskets

Feature factory PMs also have a tendency for big bets with many parts that all need to be delivered to even know if there has been an impact. Features take months to build, then get launched to the sound of crickets.

In our fictitious example, we linked the business objective to increasing the adoption of feature X, making a single bet about what would have the desired business impact. Without using a method of discovery like the Product Kata, this is more of a hunch. That’s not to say hunches are bad, but before we go spending the company’s money we should make sure we have properly explored the problem space.

In the Kata, we observed multiple things that contributed to the desired business objective. Now, if only there was a way to visualise the Product Kata and visualise the exploration of the problem space…

This segues into another neat tool: opportunity solution trees. This tool allows the team to create a mental representation of how the business objective could be achieved, while making the guesses/assumptions obvious.

Example Opportunity solution Tree

So how does it work?

• Blue: Good product discovery starts with a clear desired business outcome. For us this is defined in the OKRs for our team, as set against the agreed business objective.
• Green: Opportunities should emerge from generative research. From research performed as part of the Kata such as user interviews, surveys, feedback and the like, we produce opportunities worth exploring.
• Yellow: Solutions can and should come from everywhere (as long as they are bounded by an opportunity). Grouped together by opportunities, we can generate ideas that would have impact.
• Orange: Experiment to evaluate and evolve your solutions Lastly, rather than fully commit to a solutions feature set, we can experiment our way towards it. It’s here we combine with the Kata (Obstacles and Steps) and use the experiments as steps towards the target condition/business objective.

There are many benefits

• Promotes the idea of many bets or multi-tracking towards the business objective
• Ensures opportunities are exposed through generative research (like the Product Kata)
• Binds solutions in groupings (opportunities) that are themed, contextual and most importantly, aligned to the objective
• Visualises the team’s mental model of the problem space
• The trees can also be used in reverse; if you have a lot of ideas (or a backlog you’ve just taken over), you can work backwards to the objective and discard ideas that don’t align.
• Allows you to measure at many levels — business objective, opportunity solution and experiments

Use opportunity solutions trees to foster and visualise multi-tracking and experimentation, and to avoid single idea bets. Importantly, use as communication tool for the business and peers to explain thinking.

This tool isn’t mine, here’s the original (and better) post about it from Teressa Torres

A soupçon of process, validation and documentation

Another defining feature of Feature Factory PMs is process, or lack thereof. This includes how they define impact or validate an idea.

At Ansarada, we’re no strangers to starting with why, then how, then what. It’s one of our values (curiosity). We use this format for most of our communication as it covers what’s required to give context and direction.

This last tip is about a pragmatic application of the items discussed in this post: a one pager document for each idea.

Ok, I realise that this isn’t an earth shattering revelation, but with this simple process you can avoid so much of what happened to me early in my career. And done right, it will help answer many of these questions

• Are we building the right thing?
• Have we moved the needle on X metric?
• Did we get value for the money we spent?
• How does this relate to business objective Z?
• Why did we decide to build this?

Example One pager

Our one-pagers include the following;

Be transparent — show your working!

By including a hypothesis along with the problem, we can state what we think will happen with the introduction of this solution.

Validate and prioritise

RICE scoring, introduced by the amazing folks at Intercom, is a modification of ICE scores first introduced by growth hacker Sean Ellis.

The RICE acronym scores each potential solution along the following dimensions:

• Reach: how many people will this impact? (Estimate within a defined time period)
• Impact: how much will this impact the goal? (Massive = 3x, High = 2x, Medium = 1x, Low = 0.5x, Minimal = 0.25x)
• Confidence: how confident are you in your estimates?
• Effort: how many “person-months” will this take? (Use whole numbers and minimum of half a month — don’t get into the weeds of estimation)

REACH x IMPACT x CONFIDENCE / EFFORT = RICE SCORE

We’ve further extended the confidence estimate with an idea from Itmar Gilad, where confidence is scored based on validation tasks you perform. This is super useful, as it ensures we’re checking with the customers that we are building the right thing.

Confidence scoring

Validation tasks to raise confidence

Align to the objective and group to the opportunity

Be explicit about what each solution is going to impact and ensure alignment across the team. Each one-pager should be linked — both in concept and digitally — to both an objective and an opportunity page where both are measured.

Measure the right thing, then check if it worked

We need to define the measurement of success at multiple levels. First, at the desired business objective, then at the opportunity, and finally at the solution level. For each one-pager the success metrics should be defined; I suggest using the HEART framework and picking the 2 relevant metrics for the solution.

However, the measurement of the solution is not enough. We need to be looking at the impact to the business metric, which we record in the Result field. Obviously, this will lag after release, so we should set a timeframe to define if the solution worked or not and record it. This should be reviewed as part of the PM team. This might be painful to begin with; but ultimately will improve your product thinking and approach.

No more factories please!

I hope these tips and frameworks help you avoid building a feature factory of your own!