Jul 4, 2021
Concepts and Sample Frameworks for Software Forecasting
Part 2 of Democratizing Operational Excellence: Developing Competitive Advantage in Software Forecasting
Please read Part 1 for context.
Estimation, Capacity, Sizings, and Forecasting
You need to look back to see forward.
Estimates → capacity → sizing prediction → forecast → [iterate]
An estimation is an analysis of past actuals data that attempts to state the true nature of the team’s capacity. By looking at the statistics of delivery (work over time), an estimate can be made of what a team’s expected output might be. This gets more accurate sprint over sprint.
Capacity is calculated once there is a reliable estimation of a team’s output. It is a calculation of the average amount of work you estimate the team can do in a given time interval. Having this drives higher confidence into forecasting dates from a sizing.
Capacity = (avg size of work / time interval / contributor) * team size
A sizing is an estimation pairing rear-view capacity with forward looking work prediction including uncertainty, complexity, and effort.
A forecast is a prediction of the probability of an unseen future delivery. A forecast is a special case of prediction where time series data is used to make the prediction.
If a sizing doesn’t leverage past data, it’s not a forecast, it’s a blind guess. The only time blind guesses should be employed is with a brand new team that hasn’t delivered anything yet. As soon as a team is on sprint 2, it should begin practicing forecasting.
Story Point Sizing
Do your research on sizing approaches. Most experienced development teams have moved away from absolute-based sizings (a day, a week) in favor of relative-based sizings.
Story points are a unit-less measure whose progression is based on the Fibonacci sequence. The sizes are based on complexity, ambiguity, and effort. Sizes are relative, where a subsequent size is roughly as much work as the previous two combined (1, 2, 3, 5, 8, 13, …).
Story Point Sizing Table
Share this table with your team for use in Planning Poker, a team-oriented story point sizing activity.
Planning Poker
Planning poker is a team activity where, for each chunk of work under consideration, members spend a minute considering the work and privately guess at the story points based on the table above. It’s called poker because members are given a card with each Fibonacci number on it. I’ve used hand-held whiteboards or Slack channels instead of physical cards also. It’s important that everyone reveals their sizing at the same time to drive the greatest diversity of thinking into the mix. If the numbers generally agree, record the number and move on. If there are outliers, have them make their case, quickly evaluate the information, and move on.
For high-level sizings at a release-level, this can be done with functional leads (design, eng, infrastructure, PM). For sprint-level sizings, this should be done with the individual contributors.
Improving Sizing With Estimation of Capacity
Here we have two software teams. What can we learn from their story point sizings over time?
Team A predicts they can deliver, on average, 10.5 story points per contributor (σ=1.2, MD=1) per sprint but they deliver, on average, 7.1 story points per contributor (σ=3.5, MD=2.6).
Notable observations
- Team A consistently expects their capacity to complete work is greater than it is. Their average actual, per contributor, is several points of work below their plan. They don’t appear to be learning over time, perhaps because they aren’t looking at their data.
- Team B actuals very closely match their plan. They are learning from their past to better predict their future.
Leveraging this data to iteratively improve forecasting
- By tracking the stats on their actuals, they can feed that into their predictions for the next sprint. Team A clearly isn’t practicing this approach. Team B clearly is.
- Team A has no clear estimate of its capacity. Team B has an excellent estimate. Since the variability of actuals is high on Team A, their future predictions will likely continue to be off. They should focus on understanding the variations in output.
Confidence Level Framework
Iterate on a framework like this that fits the data your team has available
Gantt 2.0 Visualizing Confidence and Risk in Timeline
The Gantt Chart Fail
The problem with visual timelines, which are helpful visual tools in communicating a plan, is that they don’t typically convey confidence or risk. Options for this:
- Show the end date as a date interval (a range of earliest to latest probably dates) and color the bar with risk
- Include text in the Gantt bar showing confidence level and color the bar with risk
