Ditch low-level estimation sessions; use data-driven forecasting

“The Sydney Opera House was expected to be completed in 1963. A scaled-down version opened in 1973, a decade later. The original cost was estimated at $7 million, but its delayed completion led to a cost of $102 million” — but your project will be different, right?

“When’s it going to be done?”

… that’s the million dollar question.

“Estimation is easy. Anyone can do it. It’s being accurate, that’s difficult.”
— Anonymous

Many projects are long long past their estimated deadline or abandoned completely. They are grossly underestimated and are over budget. Software projects aren’t much different.

Estimation sessions and ceremonies are typically a waste of time and money. People hate sitting in them, but are often persuaded (due to perceived pressure) to go along anyway. The tools, techniques and planning games that surround the dark arts of estimation keep growing and becoming ever more superstitious in nature.

Yet we don’t seem to get better at producing estimates. In fact, studies show that we suffer from an optimism-bias, and the better we think we are about estimating, the worse we get at it. Even if people are asked to assume the project will go horribly wrong, they will still be too optimistic and drastically underestimate. It’s all part of being human.

Conversations with stakeholders aren’t always this civl — http://dilbert.com/strip/2003-01-10

If you ask someone “how can we get better at giving estimates?” you’ll hear a myriad of different solutions, ranging from giving estimates in story-points (whatever those are) or “t-shirt sizes”, to simply not estimating at all — but this approach really doesn’t help our stakeholders understand the key question of when something is going to be done.

Not only are planning sessions not helping us get better estimations, they’re wasting time and money. Even a junior agile team spending 15 mins in a planning session probably costs around your business around ~$44.

But it doesn’t have to be the case. There’s ways we can improve our forecasting using data-driven approaches. There’s also ways we can deliver the estimates to give more confidence to our stakeholders and give us more flexibility if they slip or scope is increased.

Multi-Dimensional Estimates

It’s probably a good practise to start adding extra dimensions, other than just time, to your forecasts. Think about things like: confidence level, margin of error, etc. — whatever makes sense in your context.

Consider the following alternatives:

A. “These tickets will take 2 weeks to implement.”, or;
B. “I’m 80% confident that these tickets will be completed in 4 weeks.”, or;
C. “These tickets will take around 10 days to complete, ± 1 day

The extra dimensions can help tell a better story and give you more room for things going wrong. It’s important to also re-evaluate your estimates if the original scope increases or decreases (we’ll talk about that later.)

Forecast Metrics!

If you’re in an agile team, you can start to move away relatively easily from poor estimates just by doing a few things with your existing delivery metrics:

Start tracking your current work

If you use JIRA, Trello, or another electronic board, you can probably automate this quite easily. If you don’t, just start taking taking a note of how many “tickets” per day you put in the ‘Done’ column (as in, actually done, not just dev complete.) Nothing fancy.

What now?

OK — so now you have collected some data. You’ll only need a few weeks worth at most. Now you can use a method of simulation called Monte-Carlo — it’s used by all sorts of people for different use-cases (finance, weather reports, project management) and it runs multiple simulations using your data to give you an estimate, often with a confidence level, of what you can deliver based on your historic performance.

Agilytics is a tool that can automate your data-driven estimates using existing Jira data.

There’s automated tools out there to help you with this if you already use Jira, like Agilytics. Or if you want to manually do this, there are spreadsheet templates that will help you achieve the same thing.

The book Team Guide to Metrics (Chris Young & Mattia Battiston) is an excellent resource on how you can implement this type of metric — and lots more. (Also check out Mattia’s talk which goes over this in some detail including how to use the above spreadsheet template.)

Wait, we can cheat at this!

Yes, and that’s the point — the next thing is to naturally break things down into smaller and more consistent pieces of work, which your team should naturally begin to do (and this doubles as a coaching aid!)

Then, as you go, your metrics reflect less erratically sized tickets, and your forecasts get more accurate.

Re-check your estimates. Then check them again. (And then again!)

https://xkcd.com/612/

Lastly — and maybe most importantly — it’s worth regularly re-checking your estimates. The truth of the matter is, no matter how good your forecasts are, they’re always historic. And things like scope change all the time. So if you are able to check them regularly, you can see whether you’re veering off course.

Imagine if the sat-nav gave you an ETA when you set off on a trip but didn’t ever update from its initial evaluation. If you’re going to be even 20% wrong, you want to know that now, not when you end up arriving late.

References

1. The myth of software time estimations (medium.com) — https://hackernoon.com/the-myth-of-software-time-estimations-576a7466d91a
2. Estimation (tastycupcakes.org) — http://tastycupcakes.org/tag/estimation/
3. #NoEstiamtes (medium.com) — https://medium.com/@johnpcutler/noestimates-two-different-problems-b4f37d29ce5d
4. Developers: Be Less Exact, Make Better Estimates (medium.com) — https://medium.com/@gsto/developers-be-less-exact-make-better-estimates-d0377deb393b
5. Planning fallacy — https://en.wikipedia.org/wiki/Planning_fallacy
6. The Hourglass Is Half Full or Half Empty: Temporal Framing and the Group Planning Fallacy.
   Sanna, Lawrence J.,Parks, Craig D.,Chang, Edward C.,Carter, Seth E.
   Group Dynamics: Theory, Research, and Practice, Vol 9(3), Sep 2005, 173–188 — http://psycnet.apa.org/doiLanding?doi=10.1037%2F1089–2699.9.3.173
7. You waste a lot of time at work (Atlassian.com) — https://www.atlassian.com/time-wasting-at-work-infographic
8. Your scarcest resource (hbr.org) — https://hbr.org/2014/05/your-scarcest-resource
9. Estimating the Cost of a Meeting (hbr.org) — https://hbr.org/2016/01/estimate-the-cost-of-a-meeting-with-this-calculator
10. Monte Carlo Simulation (palisade.com) — http://www.palisade.com/risk/monte_carlo_simulation.asp
11. Team Guide to Metrics (Chris Young and Mattia Battiston) — https://leanpub.com/metricsforbusinessdecisions
12. Kanban Metrics in Practice (infoq.com) — https://www.infoq.com/presentations/kanban-metrics-sky