Don’t estimate: forecast! Predicting software delivery using metrics — Martin Aspeli

I was fortunate to hear a talk recently by Martin Aspeli, Head of Engineering at Deloitte Digital, explaining how and why to use metrics to predict software delivery rather than estimates. Here are my brief notes:

tl;dr: there is no need to spend time estimating — with discipline and data we can forecast!

1. Predictability is more important than pure top speed for most organisations.
2. Humans are notoriously bad at estimating. This is a facet of human nature — as a species we are bad at estimating. See Hofstadter’s Law.
3. We tend to ignore Wait Time when estimating, yet Wait Time makes up the majority of Cycle Time in most organisations doing knowledge work like software development.
4. With some discipline in how we use Jira, we can start to use metrics to help track how long things actually take — this increases predictability.
5. We need to take care to ensure that the Arrival Rate of new work items does not exceed the Departure Rate. Treat our team and its context as a system to be cared for.
6. Use a Cumulative Flow Diagram (CFD) to track the system health.
7. Measure end-to-end Cycle Time of actual stories. No need to size the stories with Story Points or t-shirt sizes. The ensemble of stories provides confidence measures.
8. As people are “loaded” beyond about 80% capacity, their capacity to respond drops dramatically. This is basic queueing theory.
9. Forecast the delivery dates for batches of work using Monte Carlo simulations. This gives confidence measures (50%, 85%, 95%, etc.) allowing stakeholders to choose dates based on likelihood and risk.

Further reading

• Thinking, Fast and Slow — Daniel Kahneman
• Essential Kanban, Condensed — David J. Anderson & Andy Carmichael
• Actionable Agile Metrics for Predictability — Daniel S. Vacanti
• The Principles of Product Development Flow — Don Reinertsen
• Actionable Agile: http://actionableagile.com(tools for working with data from JIRA and similar systems)
• Focused Objective: https://github.com/FocusedObjective/FocusedObjective.Resources (open source Excel based tools)

Slides