Product Management 2.0: A Growth Story

Are product roadmaps still relevant? How should product managers prioritize features or improvements? Can product management learn anything from the growth hacking movement?

Alex Davis
Oct 28, 2016 · 9 min read

After 4 years of doing growth hacking, I recently made the switch to full-time product management by joining both the Firefox Sync and Accounts teams.

As a reminder, growth is a combination of product and marketing so I thought “hey, this is going to be a breeze.”

The Early Struggle

It turns out that transitioning from growth to product hasn’t been as easy as I would have thought. The first thing I struggled with as a product manager was the idea of product roadmaps. I felt the obligation to create one since… it’s what product managers do. Right? But committing to 3 months of features felt like a huge constraint. How can you be agile with that? What if new opportunities come up? What if we need/want to make improvements to something we just shipped? Does it push our whole roadmap off? Will we seem unreliable? How do we communicate this? These are merely a few of the questions that ran through my head.

To cement our roadmap even more, both my teams committed to OKRs (Objectives and Key Results) in Q3. Our KRs were specific features or patches to ship. We had committed and needed to deliver.

KPIs

As a product manager, I’ve realized that it’s really easy to get caught up in trying to ship more features. However, it’s important to remember that each feature or patch you ship is meant to move a KPI (key performance indicator). Here are some examples:

  • Improving your sign-up flow should lead to more new active users.

So, how do we efficiently impact KPIs if we commit to a roadmap? In my growth experience, the first version of anything rarely has the desired impact. If everything always worked out as expected, I would be an oracle. Unfortunately, it’s only through rapid iterations, testing and measuring that we can learn and start to move the needle. The growth hacking movement has known this process for years since the only assumption they ever make is that most of their hypotheses will fail.

Let’s set aside OKRs, KPIs and other business acronyms and talk about the approach to making measurable impact.

The Growth Process

Scientific Method

When I worked at ViralNinjas and SociableLabs, my team and I developed a growth process to prioritize by impact and document our learnings (which I thought was revolutionary). Turns out, Brian Balfour came to a similar process of his own and describes here (worth watching!) better than any of the talks I’ve done on the topic. I will only attempt to summarize it here.

This process isn’t too foreign to Mozilla. I implemented it within two weeks of joining Firefox’s growth team and its success has resulted in a gradual adoption by some marketing teams and in part by the Android and the Firefox Accounts team.

So what is this process anyways?

The process approaches improvements or new features using the scientific method. Everything is an experiment with a control. (often called A/B tests)

Scientific Method in Growth and Product Management

I create a document with the outline of all of this before starting a test so that my objective is clear. The Firefox growth team calls this document a “recipe” because it should be detailed enough that all the ingredients are there for someone to reproduce the test. We want recipes to remove any doubts around results when someone looks back at them 6 months later. They’re great for accountability too. Best of all, past recipes are one of the best sources for new strong hypotheses.

Prioritization

Before creating your recipes, how do you know which idea or problem you want to solve will have the biggest impact? Each growth idea someone comes up with needs to be added to a backlog that is prioritized with a score. This will create a product pipeline that will live and breathe and continuously change. Scores are determined by:

  • Traffic (1 bad — 5 good)
    How many users will see it? Is the change hidden in the settings or on your homepage?

Merritt Aho reminded me at the 2015 Opticon of the importance of a solid hypothesis (confidence). He managed to quantify something I knew to be true from experience. The likelihood of your test winning depends on the origin of your hypothesis. He categorized product and test ideas into the following sources:

  • It’s best practice: You have 30% chance of improving a KPI with your test/feature.
Merritt Aho at Opticon 2015

Let’s be honest, A/B testing isn’t an excuse to fail. We’re paid to win! Back your feature ideas with great data and observations to set yourself up for success as often as possible. This is why the “Confidence” is so important to me.

The above process has now become so widely adopted in the industry, Sean Ellis’ GrowthHacker.com recently released Projects to help marketers adopt it.

If you’re just starting (e.g. startup or new product), you often don’t have the luxury of lots of metrics. This highlights the importance of prototyping early on. Prototypes allow you to collect data as early as possible through metrics and user testing.

OK, time to get back to those acronyms. Let’s try to tie those OKRs back into the mix now that we outlined the process that leads to the biggest impact and to improve our KPIs.

Combining Product Management & Growth

If you’ve made it to this point, now the juicy part begins. You’ve caught up to where I am with my own product teams today.

Here are the rough steps I’m taking to maximize product impact (move my KPIs) but to also commit to quarterly OKRs:

  1. Define and measure your company and product KPIs. Without proper data and defined success metrics, you’ll continue to ship features tied to a roadmap without any considerations for impact.

Let’s Wrap This Up

Boom! The traditional product roadmap is dead. Long live the feature pipeline. This will allow us to focus on improving our product KPIs. And to do so, we’ve moved from OKRs focused on shipping features to OKRs that care about real results.

I’m in the middle of rolling this out with the Firefox Accounts team so I will try to write a blog post with the problems we encounter as we move forward. I don’t doubt that we’ll iterate our process as we do our features.

What Are The Limits of The Feature Pipeline?

There are still things I struggle to resolve. For example, users and companies like to be able to see a product roadmap. It gives everyone something to look forward to. A pipeline makes that a lot harder. I am left communicating my objectives rather than my features which have been reduced to tactics. But perhaps users are getting used to that. Most top mobile app makers have reached a point where they no longer even write release notes with each update. Has this become the new norm? Are objectives enough product vision to please managers? What are your thoughts?

Another problem that some teams might face with a process like this is related to cycles. If you work on long release cycles (4–6 weeks in a nightly build + 4–6 weeks in an Alpha build + 4–6 weeks in Beta), your quarter will be over before you even know if you managed to achieve your key results and there’s no way you will manage to iterate quickly. If you want to make any changes after your feature is released and you’ve captured relevant data, you will need to wait another 3 months. Beyond the delays around iterating, I can see how one could get frustrated with KPIs and OKRs focused on results since it becomes so hard to move them. You’d nearly have to nail things perfectly on the first shot, every time.

If only we could all “Go Faster”. ;)

This is my first Medium post so I would certainly appreciate the love if you could click that little heart ❤. Please share your comments and feedback too.

Startup Grind

Stories, tips, and learnings from and for startups around…

Startup Grind

Stories, tips, and learnings from and for startups around the world. Welcoming submissions re: startup education, tech trends, product, design, hiring, growth, investing, and more. Interested in submitting? Visit our submission form here: https://airtable.com/shrShpeN89HrzCzOB

Alex Davis

Written by

Product Manager on @Firefox Accounts & Sync at @Mozilla. I love exploring the intersections of product & marketing. Formerly #mobile #growth at @Firefox.

Startup Grind

Stories, tips, and learnings from and for startups around the world. Welcoming submissions re: startup education, tech trends, product, design, hiring, growth, investing, and more. Interested in submitting? Visit our submission form here: https://airtable.com/shrShpeN89HrzCzOB