What I’ve Learned About Product Roadmaps in the Last 6 Months

I’ve been involved with product roadmaps for the last fifteen years. Yet I’ve probably learnt the most in the last 6 months.

Most of my experience has been working on multiplayer games (RuneScape, FightMyMonster, Moshi Monsters) with either a subscription or hybrid payment model.

Product roadmaps for subscription games are normally planned out well into the future — often a year ahead. This is necessary for a few reasons.

Firstly, large updates can often take 6+ months to build, so you’re literally forced to think that far ahead.

You normally have to publish a steady flow of varied content. The only way to be consistent with that, is to be super organised and plan well ahead.

Lastly, players expect to know what content is around the corner. Giving them visibility of a strong pipeline of content ahead also helps retain subscriptions.

For RuneScape, I had the following rules:

• months 0–3 — weekly updates should be locked down and unlikely to change (most are being worked on, or ready to go).
• months 3–9 — each week should have a specific update planned, but it’s ok if things move around a bit. Not too much though.
• months 9–15 — super fuzzy. Often working titles and things will move around and disappear a lot.

The above allows for there to be some thought and structure to how the months and quarters hang together. This helps ensure whatever a players taste, there is good value each month and quarter.

The only other rule was there should be at least a four week backlog of completed work at the front of the roadmap. Finishing things at the eleventh hour = stuff going wrong.

Building a product from scratch

The above approach works well for a big, well established MMO with a development team of well over a 100 people.

Earlier this year, I got stuck into building a product from scratch (Rescover). I was soon to learn that my usual approach to product roadmaps would be challenged.

Now that I look back to when we started Rescover, I can connect the dots and they make a lot of sense. In fact, I’m proud of what we’ve tackled, and in what order we tackled it. We’ve been very focused.

I don’t think I would have been able to imagine and plan out that journey at the start. When you’re building something from scratch, you need to be more flexible and have a shorter-term view. You learn a lot as you do things and that often influences what you focus on, or build next.

I’ve had to let go of the desire to plan so far ahead.

That said, on the flip side, it’s dangerous to just do what you feel like doing at any given moment. There needs to be some logic and structure to how you move forward in the short to medium term.

How do you balance having a plan and taking things a feature at a time?

Looking back, our roadmap was always guided by a phase or an area of focus. Whilst we had a flexible approach to what features we built, they always fell into the best things to do, for whatever the focus at the time was.

We tended to focus on a product build at a time (we shipped a release about every 3–4 weeks).

Once we felt we had done enough for a particular phase, we would then naturally shift on to another phase.

With hindsight, everything we have done so far has been guided by three clear phases. We’re now embarking on the fourth.

The three phases we’ve worked on so far are:

1. Build first version of product

We had an idea and to realise it, we needed to build a bunch of things which would form the core of the Rescover app.

They were the content feed, release detail, following, my releases, search, settings, analytics, admin system and last, but definitely not least — content.

Upon reflection we could have built less and shipped Rescover earlier. But I couldn’t be more proud of what we released. It looked nicer and worked better than what most funded start ups with full time teams are able to achieve.

Release detail page (left) and my releases (right)

2. Retention / engagement

Now that we had the basics in place, and some numbers to look at, we decided to double down on retention.

What it boiled down to was, we needed people to follow things. If people followed things, it meant they got what the point of the product was. It meant they got some early usefulness out of it. Only after a follow does the product start to deliver its core value. (my releases gets populated, notifications about when things come out etc.).

Working backwards, it was obvious to us that follows are driven by being able to find things you like. Either things they already knew about and liked — or new things that they didn’t know about, but might like.

So, the focus became one of discovery. We had to make it WAY easier to find things you like. Once you found something you liked, this would turn into a follow. And then the product would deliver it’s value.

So we decided to build collections, tags and completely re-worked the navigation (ditching the ‘hard to grasp’ filters with it).

An example of how we feature collections within a content feed (Hot in August)

Whilst not tying directly into making it easier to find things, we were also feeling an itch to improve our notifications. They are such an important part of what comes after a follow.

Quite frankly, we supercharged the shit out of them. We went further than most normal people would.

We went from notifying users only about when something came out, to when new trailers arrived, dates had changed and new releases for things you’re following are announced (sequels etc.).

We also built a system to be able to customise the push notifications that went out. So our pushes went from being automated to personalised and finely crafted. This took quite a bit of dev and content work to pull off and maintain, but we thought it was worth it.

Lastly, during this phase, we picked one metric to measure our results. We decided on how many releases people followed in their first 7 days.

This is similar to Facebooks 7 friends in 10 days metric in the early days. This helped keep us focused on only building features which would impact this metric.

3. New user retention

This overlaps with the previous phase, but with a more intense focus on the first 1–3 days of a new users experience.

We stumbled into this focus from reading Andrew Chen’s article New data shows losing 80% of mobile users is normal, and why the best apps do better. It definitely planted a seed for us.

What really did it was a conversation with a very smart friend, who drove a specific point home.

The main difference between the top 10 and next 5000 apps D30 and D90 rates, was their D1 rate.

Everyone needs a few smart friends like this. Most conversations I have with Chia Chin, I tend to feel stupid at several points. But that’s ok, because it means I’m being taught something each time!

It’s astonishing to see the D1 and D30 relationship on a graph:

image borrowed from Andrew Chen’s great article

So, we decided to focus on improving our D1 > D3 rates.

We felt our best shot at making a dent in the D1 > D3 rates was to proactively reach out to users in the first few days.

But with what?

A content suggestion seemed the most obvious and useful notification. It also neatly tied into our goal of encouraging people to follow things.

So, we set about building a system which would build a taste profile of a new user, based on their initial following behaviour.

We would then reach out to a user on D1, D2 and D3 with (hopefully) interesting and personalised content recommendations.

I’ll expand on how we did this exactly in a future post. There is some pretty interesting and clever stuff going on behind the scenes.

This feature is actually going live today which is exciting!

So what is the next phase / focus?

If we reflect, to date, we’ve built a damn fine looking and useful app. We have a bunch of power users and pretty good retention on the whole.

The next stage is organic growth.

We’ve always focused on growing by at least 8% each week. We took this lesson from Paul Grahams excellent article on Growth — Startup = Growth. We also made sure to avoid vanity metrics, so we chose weekly active users to measure growth.

We’ve bought a lot of that growth with paid installs. Don’t get me wrong, we’ve also seen some nice organic growth, the two combined allowing us to keep up with the 8%.

However, it recently hit home that if we are to sustain this level of growth and support an advertising business model, we need to be VERY big.

And you can’t buy your way to VERY big.

We always knew this, and to our credit we stayed focused on the right things to date. Building out a core product and doubling down on engagement and retention.

It was working on a growth model, that made it super clear we had to grow more quickly, and specifically through organic channels. If we were to get into the millions of active users, we absolutely had to have a certain level of organic growth.

(Note: I might write about how we developed our growth model in a future post — and even share it if people are interested?)

So, with that, the next stage for us is growth — specifically organic growth.

We’re going to initially focus on making sharing more obvious and slicker (we have some pretty awesome ideas on this).

We also want to curate news and tweets for items you’re following. We plan to provide that in a dedicated ‘buzz / news’ type of feed.

The curated news is actually a retention / daily engagement piece we’ve been itching to get round to. It’s a first step to creating a daily habit in Rescover, something we’re missing. Until now, it hasn’t quite got into the highest priority bucket.

The reason we want to tackle it as part of the organic growth focus, is we think curated news for items you’re following is very shareable.

The daily habit is a double win for us, hence our decision to prioritise this immediately after our first set of sharing improvements.

Ok, so what am I saying with all of this?

This post has been a bit of a stream of consciousness. But, I think it points out some good lessons for how to think about and manage roadmaps, which I had better summarise before I wrap up.

1. Let your roadmap be driven by a phase or focus at any given time. The phase or focus should drive decisions for what to build and in what order. Make sure everyone working on the product knows what it is and is bought in.
2. Lock down a product build at a time and then intensely focus on that. Avoid getting dragged into talking too much about other things. Definitely avoid tinkering around and building other things.
3. Have a review period after each product build. Nothing formal, just take a breather to reflect on where you are. Do you need to build more features for the phase you’re in and if so what should they be? Or is it time to shift focus?

I’ll finish with an observation that just came to me.

Don’t worry too much about what or when the next focus is. When you need to shift focus, you’ll always feel a pull to some place else. It will usually slowly build over a number of weeks.

This can be sparked by a conversation, an article, seeing some numbers or maybe just a growing feeling. When it happens, you’ll find yourself starting to think about the best point to wrap up the current focus. And then, what the first product build for the next focus should be.

If you’d like to be notified by email when I write something new — sign up here.