Velocity is to impress. Use Lead Time for predictable outcomes.

“Since the past three sprints, we have improved our average velocity by 15%.”

It was a moment of pride. At that time, I was ignorant of the prowess of the Lead Time. I relied on the metrics of velocity to impress myself, impress my leaders, and motivate my team. Whenever CEO asked me the dreaded question — “When will it be delivered?” I always hid under the veil of “not yet estimated.”

Photo by the Author representing exact feelings to the CEO’s question

I had a standard reply — “Once I have the story point estimate of all the scope items, I can tell you how many Sprints it will take to complete this work. Our velocity is 42 Story Points per Sprint, 10% higher than the previous. So we are improving.”

Put yourself into the shoes of that CEO looking out for an answer to help them prepare for the next board meeting — a response like the number of days so they can strategize their business goals.

I asked myself — what is that number? Where to find it? Should I divide the number of story points by the average velocity to derive the number of Sprints? Are Story Points additive? Can a relative number be divided?

Velocity Candle Stick Chart

I took a step back to find out how stable our velocity was. It was a good find. I plotted the data using a candle stick chart.

Candle Chart showing Average Velocity variations over 4 Sprints

Reading the chart gave us the following inputs:

1. Which teams have fewer variations in velocity?
2. Which teams have more variations in velocity?
3. What is the root cause of more variations in team velocity? The reasons like attrition, the absence of a Product Owner, and other qualitative causes.
4. It helped us to create some action plans to reduce average velocity variations.

But “that number” the CEO was asking for was still missing.

Correlating Story Points with Effort

Another idea popped up, why not find the relationship between Story Points and Time spent to complete the scope item/user story?

But there was some uneasiness inside me like I was trying to fool the system. Is finding a correlation between story points and actual hours spent reasonable? Can we correlate a relative number with an actual number?

I did some research and found that people are doing it.

Reference 1: Create sizing table. https://www.parabol.co/blog/what-are-story-points/#:~:text=The%20concept%20of%20story%20points,use%20other%20frameworks%2C%20like%20Scrum.

Reference 2: Story Point estimation cheat sheet https://teamhood.com/agile/story-point-estimation-table/#:~:text=Story%20point%20estimation%20is%20a,in%20accordance%20with%20that%20task.

I also did it but found no correlation. Here, have a look at the “story points” vs. “time taken to finish” spider charts. There is too much variance.

Spider Chart depicting the correlation between Story Point and Time Taken to Finish

1. A user story with 1 Story Point took anywhere between 1 day to 7 days.

2. A user story with 2 Story Points took anywhere between 1 day to 10 days.

3. And likewise.

4. It was all very confusing.

So I ran to Ron Jefferies, the creator of Story Points, and found this-

https://ronjeffries.com/articles/019-01ff/story-points/Index.html.

This article starts with

“I like to say that I may have invented story points, and if I did, I’m sorry now. Let’s explore my current thinking on story points. At least one of us is interested in what I think.”

This correlations experiment, too, could not help much to find the number the CEO expected from me.

These metrics based on relative story points and the velocity charts may not provide sufficient insight to manage the development and release at scale. We need predictable outcomes.

It was time. Time to look beyond

“Look beyond what you see. Look beyond your flaws and your sizes and imperfections.” — The Lion King.

Looking Beyond — Lead Time metrics

It was time to explore lead time metrics. I plotted lead time against the timeline for all the stories/scope items completed by the team over six months. This scatter plot enabled us to find the percentile at 80 and 90.

Lead Time Scatter Plot for completed Stories with p90, p80, and p67

Findings:

1. p90 was 18 days or less. We have 90% predictability in the system to complete any story/scope in 18 days or less.
2. p80 was 13 days or less. If you are willing to take a 20% risk on the predictability of outcomes, we will take 13 days or less to complete any story/scope. Alternatively, based on the past six months, the team completed 80% of the work in 13 days or less. The remaining 20% of the stories/scope took over 13 days to complete.

The CEO got the number — it’s almost 3–4 weeks.

I wanted to know the predictability of the 2-week Sprint. Surprisingly, we were committing to the Sprint Goals with only 67% predictability. And hence, we were overburdening the team. It was one reason why the last day of the Sprint was always a disaster.

The visibility in lead time also gave the team a chance to improve. The team could identify the scope items dragging the lead time from 13 to 18 days (p80 to p90). We had quantitative results and hence measurable improvement goals.

Measure what matters. This level of transparency provided by the lead time scatter plot gave insight into variability. It nourished a sense of continuous improvement and henceforth improved our business agility.

With the lead time, we answered the question — “When will it be delivered?”

W Edwards Deming once said — “Without data, you are just another person with an opinion.”

I would instead extend this — “Even if you have the data, if you cannot draw any quantitative conclusion and measurable improvement from it, you are just another person with an opinion running like a headless chicken.”

Hence, before you create any metrics, ask yourself — is it actionable metrics? Can the measurements provide continuous improvements? Can we make our team sustainable?

In Short

Today, we need to focus on the work, the flow of work, and not the workers. We need transparent and actionable metrics. Lead Time is one such transparent and actionable metric. Transparent in the way that it provides visibility in the work’s progress and actionable because it promotes team intervention to improve the process/workflow. Lead Time is more tangible, and everyone can interpret it easily. It encourages acts of leadership at all levels. We need to shift our mindset to measure the predictability of outcomes and achieve business survivability.