Flow Metrics @ tb.lx series: Discover the Leadtime and Leadtime Breakdown metrics
Welcome to Flow Metrics series by tb.lx! This is our third article dedicated to flow metrics, and the second one where we dive deep into another two metrics; Leadtime and Leadtime Breakdown!
(Read the first article here to understand more about what flow metrics are and to meet our agile specialist Breno Campos, and here to know more about the first metric, throughput).
Our second metric comes in a pair and is a great way to check if the sizes of work items are standardized; let’s talk about the Leadtime and Leadtime Breakdown metrics. Leadtime is necessary to check the predictability of the team and delivery, as well as to ensure the Throughput chart is being read correctly. The Leadtime Breakdown provides fascinating insights into the process and helps us identify bottlenecks. In general, Leadtime measures the time each item spends in the workflow. This is very relevant information that you need to know since no customer will order something without knowing when it will be delivered. Would you order something on Amazon without knowing when it will arrive? Even when we order a pizza, we expect an “estimated” delivery time, right? This is the Leadtime; the amount of time it takes from the commitment point (when we order the pizza or the parcel on Amazon), until the delivery.
For us, “Leadtime” answers the question “How long do I have to wait until an item or service is delivered?”
As different stakeholders are involved in the order process, there are different needs in terms of time for each one of them. In the software context, you can mainly distinguish two points of view: the customers and the system’s.
The Customer Leadtime/ Order Leadtime starts with the customer’s order and ends with service delivery.
The Process Leadtime/ System Leadtime focuses on the internal delivery process of the service. It starts when the item enters the development system and ends when it is moved to Done. The idea is to understand the batch size of the team’s work. You can use the System Leadtime to track your teams’ predictability to better understand their behavior.
It’s important to understand the difference between these points of view, especially inside big companies, where we need to follow some steps to deliver things in production.
In addition to these definitions, you will likely come across the term Cycle Time. To better understand the difference between Cycle Time and Leadtime, let’s use the example of a truck factory working with the principles of piecework. This imaginary factory consists of 10 workstations and every truck spend 30 minutes at each workstation. At this rate, the factory will finish a truck every 30 minutes, so its Cycle Time is 30 minutes. Meanwhile, one truck has a Leadtime of 5 hours (10 workstations of 30 minutes each, totaling 5 hours). The biggest difference at play is that the Cycle Time measures the time a process takes to produce one item (like the delivery rhythm or cadence), while the Leadtime measures the time one item needs to go through the process.
Now that you know what you are measuring, you can look at how to visualize it and why it is advised to do it in a specific way.
Currently, we are considering three different ways of visualizing Leadtime. This might seem like a lot, but we can only highlight the things we want to (without overwhelming the reader of the chart), by having different metrics.
First, let’s start with the scatter plot chart.
>> For analysis: using a graph with more points
Let us guide you through it:
The blue line represents our trend of delivery. Each dot represents one single work item that has been delivered. The Y axis represents the Leadtime, while the X axis represents the items.
The green horizontal line represents 85% of the sample, meaning that 85% of the delivered items took 12 days or less to be delivered.
The red horizontal line represents 75% of the sample, meaning that 75% of the delivered items took 8 days or less to be delivered.
The purple horizontal line represents the median average of the sample, meaning that 50% of the delivered items took 6 days or less to be delivered. At tblx, we measure only work items (for Jira tasks and stories — since they have the same granularity level). Epics are not measured in this graphic, and neither are sub-tasks.
Above, we see an example of a team delivery over the last 12 weeks. This helps us to understand the trends and the team’s predictability on the delivery. The items are ordered by their closing date, which means that on the right side, we can find the most recent ones and the Y-axis shows the System Leadtime for each item.
The percentiles are also part of this metric. It highlights the number of items below the percentile value (75th percentile: 75% of all samples are below this value). These horizontal lines help you understand whether your teams are predictable or not. The closer together the lines on the graph are, the more predictable the team. This means that the team spends more or less the same amount of time on each item and that they have standardized the size of an item very well. The information we are gathering with the percentiles could be used to forecast the probability that something will be delivered by a specific date. This is a powerful tool when it comes to committing to a timeline or a delivery date. The massive difference to other approaches is that you can now use real data and a scientific approach to predict the delivery and not use your estimation as Story Points to commit to a delivery date. At tb.lx, we prefer using the 85th percentile to calculate a delivery date, because we believe it already gives us the best reliability. The percentiles also help us to consider the outliers, since life happens and sometimes, we may have delays on some items. As a result, we have a 15% buffer above our sample. Imagine that during our development flow, someone from another team or a stakeholder approaches us asking for something to be delivered in 5 days. Using the sample that we have, alongside data-driven decision-making, we can be assertive to say that it’s impossible because less than 50% of the items we delivered took 5 days or less. In this case, we can negotiate and ask for more time, highlighting the data behind our estimation, and being transparent with the partner.
On the other hand, if someone asks for delivery in the next two weeks, based on our data and performance, we have a good chance to deliver on the request, because our 85% line is at 12 days. While these numbers don’t guarantee delivery, they do give us good insights. Product management is all about managing expectations and aligning the delivery accordingly. If something goes wrong, we can contact the requestor of the delivery, to explain that the item is an outlier and that we’ll need more time to deliver it. My main advice here is to always manage expectations as soon as possible, and not wait for the final deadline to let a stakeholder know that delivery is behind.
The key takeaway for Leadtime is to use it to understand the current size of each item and to monitor if this changes over time. It provides essential background information for the Throughput. But never force the team to decrease the Leadtime — do never use it as a goal. Since we decide to use Leadtime as a goal, the teams will simply fool us with the data, slicing the items in pieces that just do not make sense just to fulfill vanity metrics. Leadtime is a means to understand flow health, not a goal to look good.
Leadtime Breakdown
>> For analysis: Use a graph with more columns
This chart is set up similarly to the previous one we showed for Leadtime. It is ordered by the closing date and the total height of each bar shows the System Leadtime. However, this chart provides some additional information.
Besides the total time the item has spent in the workflow, we can also see how much time the item has spent in each workflow step. This is very helpful when we try to identify bottlenecks or to understand which step of the workflow needs more attention and support.
Looking at the metric for the first time, we see teams spent longer reviewing the items than they would have liked. As a result, we implemented mob reviews to support each other with difficult Pull Requests (PR) to review. It’s important to understand that the more items we have on the review status, the longer time we’ll take to deliver. The amount of Work in Process affects the Leadtime. The more items we work on simultaneously, the larger the Leadtime will be. The use of metrics is not just for internal improvements, but also to make the team’s work transparent to everyone outside of the team.
With this example, we understand that the amount and quality of data available limit the power of metrics. If you want to use metrics for your teams, you should ensure that you collect the data for at least four weeks, preferably six (in our experience, a month and a half gives good predictability and shows the team’s current status). Only by having a big pool of data for each team, can you compare the current situation to previous ones and evaluate current trends. However, you should know that teams are constantly facing changes. As a result, this means that when a team applies metrics, they cannot use any data without considering the context, since everything impacts the metrics. If we use data, we must ensure we have a full picture of the situation. For a more in-depth analysis to understand trends or retrospectives, a good range is to set the data from 12 weeks ago to the present, today as it is easier to remember everything that happened during this time. This timeline also but still provides enough data to work with.
This series of articles were written with the support of Breno Campos, our Agile Coach, and Benedikt Heinz, a dual industrial engineering student at Daimler Truck and on an international assignment at tb.lx.
