On estimation: why software projects are like car journeys, and how to cope.
“ARE WE NEARLY THERE YET??”
If you’ve got kids and have ever been on a long car journey with them, that phrase will probably sound annoyingly familiar. Maybe you even remember saying it yourself when you were younger.
How about this phrase: “Is that ticket done yet?”
Or this one: “When will your project be finished?”
Or this one: “Can you tell me when the next release be shipped?”
All these questions boil down to the same thing: “How long will that chunk of software development take?”
As product managers it often isn’t our responsibility to provide an answer, but we’re frequently asked the question by our superiors or dependent teams nonetheless. And on the flip side when we’re weighing up our roadmap options, we might be the ones asking, so it’s prudent of us to understand the question more deeply.
Whether asker or askee, the answer given is usually very well meaning but ultimately is a big fat guess. And when guesses turn out to be wrong it makes people sad.
In my past life as a software developer I’ve often tried to explain why estimating software development projects is Hard, but I never seem to get through.
Well, this is my best analogy yet, so let’s see if it sticks. What follows is a musing on why software estimation is so Hard, and what we can do about it.
So, back to the question at hand: “Are we nearly there yet?”
A common answer a parent might give is: “No, there’s another two hours to go”
Ten minutes later, the child asks: “Has it been two hours yet?”
But it’s not their fault! Just as a 40 year old executive can’t understand why they got a 60% confident 8 week window as the answer to their simple question, a 4 year old child can’t grasp the passing of time and the concept of an hour.
To understand how hard it is to estimate software development, consider for a moment how we estimate car journeys…
Let’s say that you live in Cardiff, and you’re visiting friends in Leeds. A distance of around 230 miles across the UK. If your friend in Leeds says to you: “What time will you arrive at my house?”, what answer do you give?
I’m willing to bet you won’t say: “I’ll arrive at exactly 3:07pm”
You’re more likely to say: “I’ll arrive around 3pm, give or take 30 minutes” (a 60 minute window)
Or if you’re more cautious: “I’ll arrive sometime between 2pm and 4pm, depending on traffic” (a 120 minute window). Note for a journey which takes roughly 4 hours on average, you’re giving yourself a 50% margin of error!
But why can’t you give a precise time?
- You’ve driven a car before, so you know how to do that.
- Lots of other people have driven from Cardiff to Leeds, so it definitely can be done and it can’t be that hard or so many people wouldn’t have done it before.
- And you’ve driven from your house in Cardiff to another friends’ house in Manchester before, which is sort of a similar route, so you’ve got some experience in the area.
But despite all that, you can’t give a more precise time? Why not?
It’s simply down to the unpredictable nature of car journeys.
Let’s review the litany of ways your car journey could be waylaid. Many of these will be far too familiar to you…
- Roadworks. If someone has dug up the road you’ll be stuck in queues. Or worse your road may be closed completely, meaning an unexpected diversion.
- Accidents. You can’t help someone else crashing into you, or having an accident yourself, or getting stuck in the queues that develop after someone else has had a collision.
- Breakdown. Despite servicing your car regularly, it will break down randomly and require an unknown amount of time and money to fix.
- Wrong turns. The shortcut you thought you knew, turns out to be a dead end. Oops.
- Bad weather. Thunderstorms, snow, sleet, ice on the roads, floods — all will mean you have to go slower and may force you to take a different route.
- New road layout. The thoughtful folk at the local council realised that only 40% of their roads were one-way, so to help everyone out they’ve rebuilt another 25% in the most complex structure known to mankind, just to make sure that two cars never pass each other while travelling in the opposite direction. For some reason. This confuses you and you have to pull over for an unspecified period of time to calm down.
- Difficult terrain. For part of the journey you have to go off-road, through a river, up and down some unbelievably steep hills, navigate round a herd of sheep and dodge oncoming cyclists at the same time, and then pay exorbitant toll charges.
- Traffic. The sunniest weekend of the year so everyone rushes to the beach in a mad panic.
Now some of this could be taken into account when calculating a journey time estimate, but without precise details of exactly which of these things will happen and how they will happen, it’s impossible to be precisely sure about how they will affect the overall journey time.
And exactly the same is of software projects.
Let’s review that same list of things but from a software development point of view. Hopefully you can see the parallels…
- Infrastructure changes. The platform team decided they need to upgrade all the network cabling this month, periodically taking down access to Github, NPM, the interweb and so on.
- Accidents. Somebody deleted the database, committed the wrong branch and fried the live site, or spilled coffee on the only working backup server. Now it’s all hands on deck to get that fixed.
- Equipment failure. Your laptop, the servers, software licenses expiring, configuration getting lost, human illness, team members quitting, the list goes on.
- Design choices. One day we feel like we’ve made a fantastic design choice taking into account everything we know. The next day we learn something else — a new requirement or information about a better design pattern — and voila it’s back to the drawing board.
- Noise. Spurious requirements, BAU work, overhead in the form of meetings, highly urgent and critical mini-projects that get lobbed over the fence by the C-suite.
- Technology upgrades. A new version of your favourite programming language, IDE, server environment or build chain has just come out and so everything needs upgrading. Helpfully none of it is backwards compatible and there are several major new features which you have to get to grips with.
- Complexity. There is always hidden complexity in a system as we discover unplanned interactions between components and as requirements unfold into beautiful chaos butterflies. On the face of it the feature seemed simple, but once into the nuts and bolts it’s actually really hard.
- Resource overload. If 15 teams are pushing 3 build jobs each through your one puny build server, and smoke starts coming out the side of the box, you’re in trouble.
I could go on, but I think you get the picture!
As a society we all understand that car journeys cannot be precisely and accurately estimated so we don’t press people for more specific answers. We just accept it.
That’s because we (mostly) all have first hand experience of how tricky it is to estimate a car journey, so we get it when someone else can’t be sure because we wouldn’t be sure either.
The same is not true of software development.
Someone who’s never done software development before may have trouble accepting a fuzzy estimate because they haven’t experienced all the things in the list above.
But here’s the thing…
Sometimes the business needs an answer. A proper answer. One that can be used to justify and commit large sums of money, mobilise resources and used to plan and coordinate larger programmes of work across many teams.
So, sticking with the car journey analogy let’s explore strategies we already unconsciously employ to provide a workable answer:
- Set off early (effectively giving yourself buffer room before the planned deadline)
- Use the worst case scenario (say “I’ll arrive by 5pm at the latest”)
- Lots of research ahead of time (check the weather, roadworks, motorway traffic cameras, service the car — all of this takes time!), then adjust the estimate if necessary
- “Plan B” (what happens if my original estimate slips?)
Most of these strategies are about letting the other side plan appropriately. If you’re expecting someone to arrive at 3pm, and they don’t — it’s pretty frustrating isn’t it?
All of these have parallels in software, they are all valuable and can be boiled down to 3 steps:
- DISCUSS BEFOREHAND: “Weather is supposed to be bad isn’t it? Yeah? I saw some roadworks too. Sigh.”
In software this means highlighting and discussing all the things that might trip you up. Pay special attention to the things that look a bit thorny already, and at least mention the things that look fine now but which might blow up at some point. Make sure you do this AHEAD of those things actually becoming a problem.
- AGREE CONTINGENCY PLANS: “If I’m late we’ll meet directly at the theatre instead.”
In software this means having a backup plan for the project going over the deadline, and also coming in under too. You could line up extra bonus ‘nice to have’ work, or use a MoSCoW rating system to push non-critical work into the danger zone at the end. You should also prepare mitigations in case certain Bad scenarios occur. Make all of this well known to all parties.
- REGULAR UPDATES: “Roadworks were OK so I got back on track, then the world’s biggest hailstorm started and I ended up in a ditch so I’m still going to be late.”
In software this means a regular drumbeat of updates. Every week, every month, whatever suits. Each time you present progress or a projection, note when that projection will be updated next and explain why it will be better (more precise/accurate). When you do come back, explain again what’s improved about your knowledge of the project and where you’re able to be more precise/accurate. Even if no-one is asking.
An one more extra key for software:
- SHOW YOUR WORKING: Have a consistent and transparent approach to producing an answer which highlights the major causes of uncertainty, explains why the estimates are fuzzy and also explains HOW fuzzy they are. Be factual, not opinionated. Be honest about what you don’t know, and have a plan for finding out. Your stakeholders should understand your methodology as well as you do. Once they’ve agreed on the process, they can’t argue about the results that the process produces.
So what does this mean for product managers?
If you’re the one being asked: first hand off to your project manager, but if you don’t have one then use this analogy and approach around your project/delivery schedules. You can also use elements of this when sharing your product roadmap with people.
If you’re the one asking: understand that you won’t get a precise answer, and that if you do it’s probably wrong. Have your own contingency plan in place.
As for what to say the next time your child asks if you’re nearly there… well, I have no idea.