Mr Metrics: May the flow be with you
Change delivery manager Kevin Fish talks through the metrics that matter when it comes to measuring flow:
In an earlier post I’ve spoken about the importance of metrics, strike that, inspection and adaptation. If you’ve missed them, especially if delivery metrics are new to you, it’s worth taking a look: Why should you use metrics and Starting out with Agile — what to measure.
In another post, Milan Juza also talks about the importance of flow in his article about Business Agility @ BGL:
In SW delivery, flow is unfortunately a very underappreciated concept. By ‘flow’ I mean the way in which a given organisation is optimised for a fast, sustainable, repeatable, safe and streamlined delivery of value. For us, flow is one of the key focus areas for all teams and management alike. It all starts with understanding how we work.
It’s never all about one metric
Whilst individually some metrics will give you more insight than others they never tell the full story. If you’ve been around the block a few times you’ll probably have teams dashboards with x number of data points.
The story doesn’t stop there though. The real nuggets of information are in the data behind your fancy dashboards and more importantly, especially for someone like me who is very data driven, feedback from your team members. There’s no SQL you can run to get this information!
Flow
With that said, flow metrics are among my favourites to look at. Currently we are measuring Time in Process and Cycle Time with plans to extend to Lead Time in the future.
In our context, we see Time in Process (can be referred to as time in progress) as the time it takes from starting development to a requirement being ready to deploy.
Cycle Time then is the time it takes from starting development to a requirement/story/feature being live to the market/customer.
I focus on flow metrics as they tend to create the most useful debate within the team. You mini retro each story and understand how, as a team, you can adapt to get better outcomes.
Whichever delivery methodology you’ve chosen a focus on flow should not only get better results for your customer, it’ll make your team feel better. Risk will be moved further to the left of your lifecycle and you should see a steadier stream of requirements being completed/delivered.
Time In Process
This tends to be the focus for my teams at the moment, as the legacy system we’re currently rebuilding gives us less control over our releasing to live and therefore less ability to influence the difference between Time In Process (TIP) and Cycle Time.
From a high level we’ve chosen to report on the median of any flow metric. We’ve chosen this approach so, at this level, outliers don’t skew what is (or is hopefully) your norms.
The next step is dive further into the data, one of the things I like to look at is the differences in average TIP at a story point level. Of course as the story points increase, so should the TIP.
In the example below the average TIP for three- and five-point stories is lower than for two-point stories. Included in the graph is also the maximum TIP by story — clearly the average is being skewed by some outliers. In these outliers, was the estimate incorrect? Were there significant blockers (waste)? Did the story properly follow the INVEST principles or was it team organisation that lead to delays?
Ultimately your task is to investigate stories and understand what could have been done differently to achieve a better outcome. Some areas of consideration:
- Waste — I’ll talk about this in a future article, but take a look at the seven wastes of software development.
- INVEST — as mentioned above, was the story split into it’s smallest releasable/testable part?
- Limiting Work In Progress — was the team trying to work on too much at one time? This will probably come out when you look at waste.
Cycle Time
As highlighted above we are currently rebuilding some of our legacy codebase, as such a portion of change still goes through a shared regression period meaning that historically we weren’t able to release when ready.
Rather than sit and wait for my teams to migrate onto a new system we’ve taken steps to bring cycle times more inline with our TIP by working with other teams to better understand the scope, and therefore impact, of our changes and understand what level of regression testing can be completed in team to allow us to release when we want.
The point I’m trying to get across here is that we could have just accepted our legacy system as a blocker and waited to migrate to a new solution to improve our Cycle Times… but we didn’t. Don’t accept blockers. Challenge the norms, positively.
“But we’ve asked that question before!”
So what? The facts may have changed since. It might be that there are no improvements or changes needed aren’t reasonable; but it doesn’t hurt to try.
One final note
Whilst flow metrics are one of my favourites, they’re not the most important factor. For me, quality is paramount.
Go as quick as you can, but if the quality isn’t at the agreed level you’ve wasted your time. I’ll talk about that in a future article.
