Flow Metrics @ tb.lx series: meet your team’s health ally
TL;DR
Flow metrics may help us work together more efficiently, productively, and healthily. Its main goal is to measure the workflow and not performance.
Data is king for flow metrics. It allows us to forecast without estimation, showing the team’s predictability.
It can also be an ally for work-life balance and a mirror of the team’s health.
Flow metrics allow us to improve the collaboration between team members, identify and mitigate bottlenecks, and much more.
You can find flow metrics in the most mundane things such as going to the supermarket or buying online.
Metrics are everywhere. You can find them in your everyday work life, when you plan ahead your week and schedule your next tasks. Or when you shop online after work and expect your placed order to arrive within 3 days. And you can’t improve what you can’t measure, right?
At tb.lx, we use Kanban as a lean method to manage and improve our operations and implement agile and DevOps software development. The Kanban system was first developed and applied by Toyota as a scheduling system for just-in-time manufacturing and has evolved until today, being implemented all across the technology value chain.
When implementing the Kanban method, metrics will allow us to understand where we are and improve our workflow efficiently, showing us where we have waste in our system and the room for improvement. Metrics may help us work together more efficiently, productively, and healthily, since they’ll show us a picture of our current process, allowing us to understand how our work behaves inside the development system, the impact of dependencies, queue time and uncertainty in the environment. One of the most valuable insights a software team can have and that metrics provide is predictability, which is the ability to provide forecasts about the delivery that are not based on subjective estimations but on actual data and science.
For that reason, we implemented metrics in all teams and enabled them in their daily work. The “Flow metrics @ tb.lx series” is a series of four articles that talk about the metrics we are using now at tb.lx, the dashboards we have set up, and the importance and benefits of using metrics. We also share insights to explain better how to read and interpret those metrics.
If you are looking for insights and a first introduction to the domain of flow metrics, this might be the perfect article for you.
To better understand the concept of flow metrics and their impact, we talked with our Business Agility Specialist, Breno Campos. Breno has been working in tech for the past 15 years, and since 2011 he has been focusing on Agile, leading teams in software implementation using the Agile approach. You can find him at our Lisbon office and as a speaker at several conferences about Agile, sharing his knowledge across the globe. “I became an Agile coach to understand how a system of software development can be improved to help people to have a balanced work life and increase the value delivered to their clients,” he explains. In 2014, Breno discovered Kanban and the flow metrics, which became his focus for future work.
But, first things first: what are flow metrics?
In general, metrics visualize the data we are collecting automatically, in our case, through JIRA, our project management tool that we use to structure our daily work in the teams. “Flow metrics allow us to “forecast without estimation,” the so-called “predictability” of a team. This means we give the probability that something will be delivered at a defined date based on historical data. It also allows us to improve the collaboration between the members, identify and solve bottlenecks, and much more”, explains Breno, who has implemented at tb.lx four basic metrics that we have adapted to get even more insights into the teams: Throughput, Leadtime, and the Leadtime Breakdown and Cumulative Flow Diagram.
You might wonder what the difference between these metrics and other approaches. Flow metrics are all about the data, and that’s their main difference, as Breno explains: “When we are talking about forecasting the delivery, simple estimates don’t work. Flow metrics don’t consider only the working periods when the developers are coding but also all the meetings, code reviews, testing, and deployments. So, flow metrics consider all these different aspects happening across time, whilst estimates, like story points, only consider the time you spend working, so they are good for measuring complexity for refinement but shouldn’t be used to forecast delivery.”
You can find these metrics in the most mundane things of your daily life. When we talk about variability and the advantages of a predictable team, Breno gives an example that clearly shows us the importance of an efficient forecasting approach. “If you buy something online and one of the stores claims that its delivery will happen between one and two weeks and another store promises delivery between two days and six months, which one would you choose? Probably the first because the level of predictability is higher. You might wait a bit longer, but the risk of waiting six months is too high. And that also applies to our teams as software developers.”
Even when you go to the supermarket, you can find Kanban principles. “At some stores, you will find two types of checkout counters: the individual ones where you line up and your time spent in line depends on the size of the purchase of the clients before you, and then you have one line for several checkout counters that call people as soon as they are free, so the customer is divided into different “resources” to decrease the waiting time. If you think about it, this is a Kaban principle, a pull system flow, meaning you receive the customer when you are available.”
Flow metrics as a mirror to your team’s health
Implementing flow metrics is also a great way to check your team’s health. Think of it as the blood analysis your doctor asks you every now and then. Breno here is our doctor and will look into the team’s results to give us a diagnosis. Are you on the right track or burned out? According to our Agile coach, this is the most significant benefit of flow metrics: to protect the teams from unrealistic workloads. “We can, based on data, have a vision of what needs to be delivered and when it can be delivered, with data to support that. It’s not simply a hint. There is a big difference between arriving at a meeting and saying, “I don’t think that deadline is realistic,” based on a guess or having historical data that supports that decision and shows that, based on the past, we can only deliver by a certain deadline. This allows us to extrapolate the data and even forecast when we will have it ready. So, the biggest benefit is making decisions based on data, not guesswork”. On the other hand, it is not always easy to have everyone on board. Although these types of metrics have been used for a long time in factories, for example, the biggest challenge is for people to see the value in these metrics “Some people prefer estimates and not measuring, but the truth is it is way simpler to use these flow metrics than any other form of estimation because metrics use historical data while estimates are just a way to guess because there’s no history supporting it” explains Breno, which lives by a great motto “stop guessing and start measuring”.
So metrics are all about performance, right?
Actually, the main goal of flow metrics is to measure the workflow
and not performance. Some people try to use these metrics to compare teams and, apparently, that’s a no-go. “That is the worst use you can give it because each team is a complex system, and they are all different. You can’t compare apples and oranges. The main idea isn’t to measure a team’s performance but the team’s health, the process and workflow — if the team is predictable and is delivering with a good rhythm if the delivery rates are good, etc” says Breno. He explains that “flow metrics can be used, for example, to measure bugs and understand failure load, to give a clear view of how many things need to be fixed after a delivery. That helps us understand the code’s quality and the quality of what we are delivering”.
“Looking into the team’s tendency, we can also measure its health. If we have a team that delivers 3 to 5 items per week and in a specific week only delivers one item, we try to understand what happened, why did we decrease our delivery rate when our tendency was higher than that? Maybe someone went on vacation, or there were national holidays. And that leads us to something we always discuss with teams which is that we don’t need to fear the fact that a metric has changed, what we need to fear is if the tendency has changed and we can’t explain why.”
And in the same way, it can be applied to higher delivery rates as an indicator of an unhealthy workload. “If that same team that usually delivers 3 to 5 items in a week is now delivering 10, you might think that if you deliver more that is always better but that isn’t exactly true. The team might have been under pressure to deliver more; if that happens, the first consequence is a decrease in the code’s quality. So it’s important to understand what happened. Maybe someone worked more hours than they should have to meet a deadline — that isn’t healthy, and the workload needs to be reviewed. The main objective of metrics is to work smarter, not harder.”
The real impact of flow metrics
This all seems super lovely but… what really changed for tb.lx since we implemented the flow metrics? The truth is that we were able to reach a higher level of predictability, leading us to be more confident regarding specific timelines or deliveries, according to Breno. “Every forecast is based on the team’s historical data. Of course, that won’t guarantee delivery in the future — nothing does — but because we are using statistics and a factual, historical and palpable basis of what we have from the team in the past, the deliveries can be better scheduled. We can be more confident about what we are delivering, when and how much time we need.” Through data and graphics, tb.lx can understand the impact of certain decisions and even the dependencies of specific teams and plan ahead. For example, imagine that we have a team that took two weeks to build a solution, but for reasons outside of our influence (e.g. an environmental constraint or release window), the solution took another month to go to production. Our time to market was one month and a half, but during this time, one month was “in queue”, the item was somehow blocked. Flow metrics give us those insights, and the possibility of planning and even understanding the impact of queues in our system.
Flow metrics as an enabler for work-life balance
“I know firsthand the positive impact it has on the work environment and teams. It allows them to take pressure off and help people to develop and grow, giving them the power to say no and decline unrealistic timeframes,” says our Agile coach, one of the team members involved in the evaluation of a 36h work week implemented recently. “These metrics were actually used to measure the impact of having a 36h week. We evaluated this change through the teams’ tendency, analyzing the impact of working fewer hours within our delivery rate and how long it took us to deliver something. We tested it for some months and ended up adopting it in September 2022”, he explains.
During the following weeks, we will post three articles about each metric Throughput, Leadtime and Leadtime Breakdown, and Cumulative Flow Diagram, and give you more insights about each.
Keep posted!
This series of articles were written with the support of Breno Campos, our Business Agility Specialist, and Benedikt Heinz, a dual industrial engineering student at Daimler Truck and on an international assignment at tb.lx.
