Count Stories, Not Points

Just say no to Story Point

One of the most common practices of teams using Scrum is to estimate work items in Story Points and then calculate and use Velocity to help plan Sprints. The Story Point concept is based on relative estimation which can be a good thing, certainly better than estimating in units of time for complex work.

However, nearly every team I’ve worked with, or heard about, that uses points and velocity, have concerns with misuse and abuse of these values. Either management is using values to compare teams, place unrealistic “commitment” expectations on teams, or they are used improperly for forecasting far into the future.

In this article I’ll discuss an alternative method that uses actual team data rather than estimates.

This method:

• eliminates the need for gaming the system to avoid retribution
• can be used to create probabilistic forecasts and avoid the “Flaw of Averages” errors when using Velocity
• takes seconds rather than protracted conversations haggling over story point values

Lean Metrics

While there are others, three commonly used Lean metrics for software development are:

• Lead Time: Often referred to as “concept to cash”. This is the elapsed time from customer request until the value is delivered to the customer. This value is essential to setting customer expectations on turn-around times.
• Cycle Time: This is the elapsed time from when development starts on a work item until it is “Done” (typically when the work item satisfies the Definition of Done). This value can help you understand whether or not improvement efforts within your process are effective. This value can also be used to forecast single item completion forecasts.
• Throughput: This is the number of work items completed per unit of time. In other words, it is the departure rate from your process.

For teams using Scrum the throughput unit of time would equal the Sprint cadence, which is typically 2 weeks. For such teams this would be the number of work items completed every 2 weeks.

For the purpose of forecasting multiple items, this is the key metric.

Velocity Is Not Throughput

Throughput is the departure rate of actual work items from your process for a given period of time.

Velocity is an average of estimates. Velocity is made up of Story Points (SP), ranging from 1 to perhaps 13 Story Points per work item. Thus not all work items are “equal” unless you assign SP=1 to every work item. Which is essentially the equivalent to counting stories.

Let’s say the team estimates two work items at 13 SP each. They complete those two, expected to be large, work items in a two week Sprint (which is the typical duration). Their velocity for that Sprint would be 26. However, their throughput for those 2 weeks is 2.

In the next Sprint let’s say the team takes in 6 work items of SP = 3 and 1 work item of SP = 8 for the same expected velocity of 26. And if they finish everything their velocity is 26 once again. But, their throughput is 7 rather than 2 as in the previous Sprint.

I think you can see how those two measures are indeed not the same.

A Tale of Two Charts

If you’re using Scrum you likely already have a chart for the velocity of the team. Using that same data, now construct a similar chart but instead of using Story Points completed use the count of work items completed.

If your team is like most, the shape of the charts will be similar. Certainly similar enough for Sprint Planning, which should be the primary use of velocity (to load the Sprint).

Any form of estimation should be lightweight and counting stories is even lighter than velocity. With the added bonus of no more long, drawn out negotiations over Story Points.

Stop using Story Points (and Velocity)

If you’re still not convinced, create a scatterplot of the SP estimate and actual cycle time for each work item. Along the X axis show SP estimates your team uses, perhaps 1, 2, 3, 5, 8 and 13. The Y axis will show days to complete each work item.

For proprietary reasons I cannot show actual team level data but, for teams that have done this exercise, these 2 things remain consistently true:

• There is little actual correlation between SP estimate and actual cycle time per SP value. Inspect the scatterplot and you will likely determine the same.
• The worst “offenders” to the above are the lowest SP values. Work items that the team considered “slam dunks” (low SP estimate) were often not. The largest variations occurred in the low SP estimates of 1, 2 and 3. There were often huge “outliers” that were longer than the highest SP estimates (typically 8 or 13). You may have had teams ask about “re-estimating” stories at the end of the Sprint because they were far harder than initially expected. It’s the slam dunks that weren’t they are talking about (to “get credit” for harder work). Teams never ask about re-estimating down high SP estimates to lower values when the work was easier than expected.

I don’t remember who wrote this but one of my new favorite hashtags: #escapevelocity

Forecasting Using Throughput

It’s beyond the scope of this article but once you start tracking Throughput you have the ability to generate probabilistic forecasts for multiple items using Monte Carlo Simulations. You can determine the probability of completion two ways:

• How many of these items can we complete in X days/weeks?
• When will these X work items be done?

Don’t fall into the trap of using the average throughput to answer these questions. Your throughput (and cycle time) should be thought of as a “shape” or range, not simply an average.

Using the average value will get you far less accurate forecasts than possible when using Monte Carlo simulations. For a far deeper look into why this is true, along with some excellent do’s and dont’s, I highly recommend “When Will It Be Done” by Daniel Vacanti.

When Will It Be Done?: Lean-Agile Forecasting to Answer Your Customers' Most Important Question

Wrapping Up

Many teams leveraging Scrum use Story Points and the associated Velocity. Beyond the potential for abuses and misuses, these values are estimates and an average of estimates. Neither have the accuracy or precision of using actual team data. Estimation of Story Points takes time and is fraught with misunderstanding, including people who are representing one estimate that simply get tired of negotiating and “give in” for no valid reason.

Using Throughput you will be able to creating useful forecasts based on the level of predictability you desire. This is a huge benefit in environments where higher predictability is desired.

Ditch the guesses, make real data-driven decisions.

If you found this article helpful, please click on 👏 and follow me for more valuable information on Agile, Lean and Book Reviews.

Until next time!