Why Agile and roadmaps don’t mix

I’ve worked in Agile for 6 years now, having worked in waterfall at Accenture for 5 years prior to that. I’ve presented dozens of roadmaps. But now I’m on a mission to kill the phrase “roadmap”, at least internally, at work. Why? Because roadmaps and Agile don’t mix. Allow me to explain.

The visualization

Ever presented a roadmap that looks like this?

Of course you have. And when you do, your stakeholders see a Gantt chart/project plan. If you’re using agile, you probably used some rough estimation to come up with the timeline. Yet the visualization makes it hard to convey the uncertainty around the timeline. No matter how many asterisks you put on the slide, they will see this as a project plan.

The expectations

So your stakeholders see this “project plan.” What do they expect? For you to deliver on it! They judge you, your squad, your parents, your value to the company on your ability to “stick to the plan”. But is delivering features “on schedule” really success? No way. High fives for delivering software features on time/on budget should be left in 2002 with waterfall.

What happens when you’re delayed? When great business opportunities present themselves? When you realize you want to iterate more on something based on feedback? You have to update your “project plan”. Bummer. Everyone was really excited about that thing at the end of your roadmap. Clients were expecting it. Senior execs built revenue projections off it. Someone from customer service Tweeted that it’s coming. Now they’re all scrambling to save face. And they hate you, product manager, for not doing your job! 😞

And you thought it’d be OK to update the roadmap because that’s the point of Agile right?

So what instead?

I’ll dedicate future posts to this topic but here’s a rough overview of what we are trying at work instead of quarterly roadmap updates – quarterly KPI Reviews. Here’s how you can use them too:

Define your KPIs

Start by identifying the metrics that you’ll use to measure success for your product. Business outcomes like revenue/profit are always a good start. Maybe sign ups as a way to measure progress on your mission. Or engagement. Whatever it is, get consensus internally before proceeding. Without agreement, you’ll always be fighting internally about feature X vs feature Y because no one will be able to point to what outcomes you’re trying to deliver. Oh and a tip here: shoot for 2–3 KPIs max. (note the K in the acronym)

Identify what hypotheses you’re planning to test to move each KPI

Once you’ve know how you’re going to keep score, brainstorm with your squad/other folks ways that you can change your product to move the needle on each KPI. Think a new first time experience will improve long term engagement? Write it down. We use this format to express a hypothesis:

IF we (make this change/build X) THEN we will see (this KPI go up or down) BECAUSE (user research/analytics that suggest why this outcome will occur)

Prioritize your hypotheses

Try to find quick wins first – what do you think will be a light lift but really impactful? Then start thinking about what you could do in 1–2 sprints to test the idea. It doesn’t have to be building new features or restructuring your whole app. The point is to get a signal that you’re onto something before investing too much time or effort. Maybe you could send an email survey explaining the concept and ask for users to vote. Or build some buttons to track interest.

Test your high priority hypotheses

Within a sprint or two, you’ve got something in front of users to test your ideas (this doesn’t have to be a full A/B test). Now track how they’re responding by looking for changes in your KPIs. Yes, I’ve assumed you have the ability to track your KPIs easily and as close to real time as possible. Without this feedback loop you’re not using Agile the way it was intended.

Present updates on the KPIs, along with what you learned about your hypotheses since the last update

Now when you get up to present to stakeholders, all you have to do is show whether your KPIs are moving – hopefully they are and if not, hopefully you can explain what you’ve learned along the way. We use a Kanban style board to show where each hypothesis is in the life cycle:

  • To Do is the prioritized list of hypotheses (some might have an ETA on them as a nod to the “project plan” roadmap we left behind, because stakeholders of course love timelines)
  • In Progress are the tests in flight in the field
  • Done are the hypotheses where we’ve done the analysis and either confirmed or busted the hypothesis (or maybe the results were inconclusive). This is where all the learnings are captured.

Don’t get me wrong. This is a huge cultural shift. And it’s not easy. But it’s worth it, because there’s so much business value in Agile if you use it correctly.

And yes, I know that sometimes you have to communicate timelines, both internally and externally. I’ll post some thoughts on how to get clients on board with this model, as well as internal stakeholders. (Hint: they don’t REALLY want features, they want outcomes too. Alignment!)

More on this later…but would love your thoughts in the meantime.

Show your support

Clapping shows how much you appreciated Rajesh’s story.