Work in Progress and the Importance of Speed in Software Delivery

Mario Konschake
Aug 31, 2018 · 7 min read

tl;dr: In software development speed is important. Main leverages to improve speed are an increased focus, low resource utilization and the identification and removal of bottlenecks, as all of those are related to the amount of work in progress. Limiting the work in progress is the main leverage to increase speed in software development.

According to Dan Norththe goal of software delivery is to minimize the lead time to business impact. Everything else is detail.” Lead time is the time between a feature request being accepted and the moment the feature is delivered to the customer.

Dan North arrives at this conclusion by making the observation, that the only thing that matters for the customer is to know if “we are nearly there yet”. Or in other words, everything which happens before the customer sees any business impact is irrelevant for them.

While it is commonplace that faster is better, it can also be backed up by quantitative arguments.

Fast Delivery Matters

In his book on The Principles of Product Development Flow Don Reinertsen, introduces the concept of the cost of delay. It is based on the observation that the value of delivering something to the market is not equally distributed over time. As an example, please think about touch-screen displays for smartphones. Such a feature had a very high value at the beginning of the 21st century but became a table stake ever since, i.e. the competitive advantage of having a touch screen became null.

The figure below exemplary depicts the cost of delay caused by a late entry to a market. The grey curve shows the potential revenue available in the market. The blue curve shows the revenue which can be still made after a late market entry.

Don Reinertsen points out that the temporal dimension of the return on invest is usually neglected in the prioritization game, while it actually is more powerful than classical methods.

Even if the cost of delay may be hard to quantify, it’s still a helpful concept to understand the importance of fast delivery in product development.

Accelerate Delivery

After we have seen the importance of speed in software delivery, let’s look at some options to increase it. There are a variety of approaches to reduce the lead time in software delivery, some more powerful than others. I will focus on the two that I believe are the most important ones.

Focus on one Project at a Time

Let’s assume we have three projects in the backlog and six engineers in our team. Looking at the figure above we can see that we should not work on all three projects in parallel but instead should work on them one after another.

In the figure below both strategies are compared. For the sake of simplicity, all three projects are assumed to be comparable in their cost of delay function as well as in their cost of delivery.

The left panel depicts the case in which all three projects are worked on in parallel and all of them are released at the same time. This is when they start producing revenue. The bottom graph shows the total cumulative revenues which can be generated by all projects. The right panel illustrates the changes when all three projects are delivered successively. The first project will already generate value after one third of the time and there is almost no cost of delay, resulting in higher total revenues.

In summary, the total revenue delivered by the second strategy is more than twice as high as in the first case. However, the cost of delivery remains the same, only the work has been organized differently.

Please note, that so far the transactional cost of splitting projects into smaller chunks, so that they can be worked on in parallel, have been neglected. Taking it to the extreme and splitting up everything into micro batches is usually also not the best idea.

However, getting the batch size exactly right is far less important than avoiding extreme mistakes and most often splitting projects in a way that two-pizza-sized teams can deliver them in one month is a good rule thumb.

Reduction of Waiting Time

Lead time is the total time a project spends in the development pipeline. However up to 90% of that time is spent waiting, i.e. only 10% of the time is spent with planning, design, coding and testing.

This means that even if we manage to code twice as fast as today the lead time would be reduced by a few percent only. Hence all our efforts to increase the speed of software delivery should focus on the reduction of waiting time.

To understand the possible sources of waiting time it is helpful to look at another system where waiting times frequently occur: vehicle traffic.

In traffic, the analogy to lead time is the travel time from entering a highway until leaving it again. There are two main sources for waiting time in traffic, it is either caused by a high density of cars on a freeway, i.e. a high utilization or by bottlenecks like construction sites. Let look at them one after another.

High Utilization

The travel time for a single car on a highway is limited by physical or legal constraints, i.e. the speed limit. However, once traffic increases, the individual cars have to slow down to avoid accidents, but for quite some time they can still go reasonably fast. Only when the density of cars approaches a critical threshold the free flow of traffic collapses and traffic jams occur seemingly out of nowhere. Once we reach that threshold the throughput, i.e. the number of cars passing a certain point in a given timeframe collapses.

This relation between car density and travel time is illustrated in the image below. It is obvious that operating close to the threshold should be avoided at all cost, because once it has been crossed the system collapses.

The non-linearity at the threshold is caused by small stochastic fluctuations in the speed of the cars which have a negligible effect, as long as the distance between cars is large, but which get amplified once cars get close enough.

An interesting property of collapses in stochastic systems is that they are surprisingly hard to recover from, i.e. the very same randomness which causes a problem (traffic jam) will usually not be able to fix it. If you are interested in the maths behind it you can dig into the theory of random walks. However, that property also becomes immediately clear when you think about how fast and seemingly out of nowhere traffic jams or other queues occur and how long it takes until they vanish again.

To draw the line from traffic to software we can simply substitute the titles of the axes in the figure. Travel time becomes lead time and car density is equivalent to utilization, which is related to the amount of work in progress. Throughput is equivalent to the number of projects we finish in a given time and flow in the context of software development can be understood as the perceived quality of work.

In the left section of the figure below, there is very little work to do. The perceived quality of work is very low (boredom) as is the throughput of deliverables. The middle section is the happy zone, lead time is still comparatively low, the overall throughput however is high and the pace of work feels most productive. We feel the flow and we have enough slack to support others or to deal with smaller interruptions. The rightmost section is the chaos zone where there is too much work in progress to get anything done at all. The throughput collapses and operating within this zone should be avoided at all cost.

After we have understood that waiting times may occur out of nowhere when we are dealing with too much work in progress, we can have a look at the second source of waiting times in traffic, which are bottlenecks.

Bottlenecks

In vehicle traffic bottlenecks are easily identifiable by the accompanying queues. Unfortunately, queues are much harder to spot in software development. Unlike queues of cars, the queues of unfinished work are invisible. In addition, queues in software development are dynamic, while yesterday a code review may have been the bottleneck, today it may be testing. This is different in traffic where bottlenecks are rather static.

The reason why we see queues forming in traffic but not in software development is that in traffic the capacity of a road acts as a constraint and there is no such constraint in software development, i.e there is potentially no limit to the number of projects which can be worked on at the same time. However, if such a constraint is added artificially by limiting the work in progress, queues will start to form and bottlenecks become obvious.

Once we know where the bottlenecks are, the theory of constraints tells us that there is no use in producing more work upstream of the bottleneck than the bottleneck can handle and that we should focus our resources on elevating the bottleneck.

Summary

Limiting the amount of work in progress is the biggest leverage we have to get the most value out of software development. Limited work in progress forces us to focus on one project at a time which leads to faster delivery. It also ensures that we stay away from the utilization threshold which separates the high throughput from the low throughput regimes and it also helps to identify bottlenecks, whose removal will speed up delivery.

Introducing artificial constraints to reduce the amount of work in progress is one of the core principles in Kanban. Therefore, it can be a great tool to increase the speed of software delivery.

I will provide further practical examples of how to achieve low work in progress in a follow-up post.

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade