What do I need to know about the planning fallacy?

The Irrational Tech: Part 1

This is a series. Find Part 0 here.

Leunig

Everyone was in the room — the devs, QA, analysts, PMs and more. I looked over the plans and confidently suggested that given our velocity we would hit our date. There were murmurs of agreement around the room. I felt confident, we had a great team, and there was contingency.


Yes, on other projects in the past things had taken longer than expected, but this one was “different”…

Well I’m sure you can see where this going.

The client was not impressed when delivery took twice as long as planned.


The Planning Fallacy

The Planning Fallacy relates to our tendency to systematically underestimate the time & resources required to complete projects.

Is that really the mostly likely date we will be finished? Or is some kind of Planning Fallacy and Optimism Bias at play?

Psychologists Daniel Kahneman and Amos Tversky first studied and identified the Planning Fallacy. They found it to be not only rife, but very hard to avoid, even with great experience this fallacy can catch out humans over and over again.

The very aptly named Chaos report last year reported that “fewer than a third of all projects were successfully completed on time and on budget over the past year”.

There are loads of stats around web development estimates, but in suffice to say, very few projects (whatever the methodology) don’t have some kinds of cost and schedule overrun.

Why do we get it so wrong?

Here are three underlying cognitive reasoning problems and some solutions to neutralize their effect on planning and estimating in the first place. (These apply to anything from a estimating or planning a 2 years project all the way down to a 2 weeks sprint.)

1. We ignore how long it took last time…

Focalism is our tendency to focus attention only to the one task, and miss other information. We also tend to only consider the best possible outcome. We tend to disregard how long similar activities took in the past.

It’s useful to remember this information model popularized by Rumsfeld:

There are known knowns. These are things we know that we know.

We can account for these in planning. E.g. Tom the QA will be on vacation.

There are known unknowns. That is to say, there are things that we know we don’t know.

We can somewhat accounts for these. E.g. We don’t have the full set of detail about for that feature.

There are also unknown unknowns. There are things we don’t know we don’t know.

A lot of estimating is done based on known knowns. Sometimes known unknown are taken into account too. But of course the unknown unknowns are not, causing bottom up planning be often wildly inaccurate.

Try this: Reference Class Forecasting

Ignore the details of the current project, and simply rigidly drive estimates by how long it took to complete a broadly similar project in the past. This is using a “reference class”.

(Techniques like agile t-shirt sizing also work on similar principles of relative sizing.)

2. We focus too heavily on known knowns…

We tend to pay too much attention to what’s right in front of us in another aspect of focalism. Then we tend to miss key things.

Try this: PRE-Mortem

We all know about the POSTmortem or retrospective. But what about the PREmortem?

A premortem trigger “prospective hindsight” by having people imagining that an event has already occurred. I’ve found the premortem to be very useful in surfacing some more of those known unknowns on software development projects.

Running a Premortem

I’ve had some good success with this technique by getting the right people into a room and frame a discussion with a version of Gary Klein’s words

“Imagine that we are now in the future. We implemented the plan as it now exists. The outcome was a disaster. What could have gone wrong?”

This technique is really powerful because it unleashes the imagination of even the strongest supporters of the plan to flush out threats.

It also legitimises doubts — it’s a good safe place and time to raise any doubts about the plan.

The premortem always brings useful tweaks (or sometimes major changes) to plans. It’s a low effort, high-reward activity.

3. We setup incentives in the wrong place…

Before we can even consider getting more accurate estimates — we need to check — are incentives in the right place to estimate resources and time accurately?

Planning errors are not always so innocent! Is the client or superiors pushing for plans with unrealistic costs or timings?

Try this: Incentivize accurate planning

Make an effort to track how close plans were to actual. Instead of rewarding people for underestimating — try rewarding people who actually created the most realistic plans.

Good luck. Tweet me your thoughts @juliaclavien!

Check out more of the biases and fallacies here -

Part 0: Why should I be interested in rationality?

Part 2: What do I need to know about sunk cost fallacy?

Part 3: What do I need to know about groupthink?

Part 4: What do I need to know about hyperbolic discounting?

Daniel Kahneman — Thinking Fast and Slow

Julia Clavien — Cognitive Bias In Software Development