Understanding Scrum metrics and KPIs
Metrics Covered
- Velocity
- Velocity Fluctuation
- Commitment Reliability
- Capacity Utilization
- Estimation Variance
- Scope Change
- Defect Leakage
- Backlog Health
Velocity
Velocity is an extremely simple, powerful method for accurately measuring the rate at which scrum development teams consistently deliver business value. It is an indication of the average amount of Product Backlog turned into an Increment of product during a Sprint by a Scrum Team, tracked by the Development Team for use within the Scrum Team. Thus, to calculate velocity of your agile team, simply add up the estimates of the features, user stories, requirements or backlog items successfully delivered in an iteration. It should the team:
- Predicting how much scope can be delivered by a specific date.
- Predicting a date for a fixed amount of scope to be delivered.
- Understanding our limits while defining the amount of scope we will commit for a sprint.
Measuring unit = Story Points
Velocity Fluctuation
Represents the velocity variance from average with relation to max and min tolerance.
Measuring Unit = Story Points
Commitment Reliability
Commitment Reliability the ratio of points delivered vs points committed to. This is also referred to as your Say/Do Ratio. Well-functioning organizations build reliable products, and reliable products are the sum of the reliability of the teams building them.
The objective is to measure, at a granular enough level, the value to the customer, so there isn’t any question about delivered scope. Everyone needs to have a shared understanding of what was delivered relative to what was committed.
Measuring unit = Story Points
Capacity Utilization
Capacity of an agile team is an important measure that should be used to inform the team how much workload it should undertake during its sprint planning meeting for an upcoming sprint. If the capacity and workload are not in balance, then the team members should have an informed conversation with the product owner on how to achieve the balance.
The estimated capacity number is used to provide guidance to the agile team in terms of how much work (stories or features to be implemented, and defects to be fixed) can be planned for the upcoming sprint.
Measuring unit = Efforts (in hours)
Estimation Variation
Estimation Variation represents the comparison of the effort that was planned (Estimated Hours) among the actual efforts (Spent Hours) with in a sprint.
Over Estimation = (Estimated efforts — Actual efforts/Estimated efforts) * 100
Under Estimation = (Actual efforts — Estimated efforts/Estimated efforts) * 100
Scope Change
Product backlog items added or removed from Sprint scope after sprint started.
Usually scope change doesn’t represents good practice but in practical scenario it happens. It too reflects quality of backlog grooming, prioritization, sprint and capacity planning.
Measuring unit = Story Points
Defect Leakage
Ratio of number of defects attributed to a stage, but only captured in subsequent stages, to the sum of total number of defects captured in that particular stage and the total number of defects attributed to a stage, but only captured in subsequent stages.
This metric reflects testing quality.
Measuring unit = Defect count
Backlog Health
Total story points available in product backlog compared with average velocity.
It reflects quality of backlog grooming and preparedness of team for future sprint. In general, team should have properly groomed backlog for next 3 sprints at least at a given point of time.
Measuring unit = Story points
