Minding your P’s and Queues: Flow Management by Queueing Theory
Queuing theory is the study of queueing or waiting in lines. According to Reinertsen’s , Principles of Product Development Flow, queues profoundly affect the economics of product development. They cause valuable work products to sit idle, waiting to access busy resources.
Intrinsically, everyone understands the basics of queueing. Think about the last time you went to a supermarket on a very busy day. Perhaps it is the day before Christmas. You are exhausted and just want to get your shopping done and out of the way in time for the holidays. You can see the lines at the tellers. There are only 3 tellers serving the more than 100 customers available. You sigh inwardly. You know it’s going to be a long wait before you’re finally served.
If you’ve ever been in such a situation you understand the basics of a queueing system: an arrival process (how customers arrive at the queue, how many customers are present in total), the queue itself, the service process for attending to those customers, and departures from the system
Queues and Product Development
Managing queues is important for all product development teams. Unfortunately, many teams are blind to the benefits of queueing discipline and how long queues affect their key metrics: lead time, cycle time, wait time and throughput.
Queues and Cycle Time
Cycle time, a metric borrowed from lean thinking and manufacturing disciplines, is simply the time it takes to bring a task or process from start to finish.
Why does cycle time matter?
Cycle time is important because it lets you know how long your team takes on average to finish tasks. Coupled with other metrics like defect density, unit test code coverage and defect injection rate cycle time is a powerful metric as a basis for continuous improvement and process efficiency.
Wait time
If cycle time is the time it takes to bring a task from start to finish, you may make the assumption that longer cycle times means slow activities. However, usually, the problem is not with slow activities but with periods of complete inactivity. It is therefore important to also measure wait time, which refers to the amount of time a task sits in a queue waiting for something else.
Going back to our initial example, suppose the cycle time is the time between when you started your shopping and when you walk out of the door and the wait time is the amount of time you spend on the queue waiting to be served
Now, imagine you went to the supermarket, picked your items and headed for the till. You have no idea how many people were queueing before you arrived, how many tellers are at the tills, maybe, even whether the queue you are on has a teller or not. Imagine you were completely blind to the process. Imagine further that you have no idea how the next person to be served is chosen, you just wait blindly until your ticket number is called. Some days, when the queues are short, this would work out ok, you would have great cycle times because of very little or no wait times. As soon as you got to the till your number is called and you’re served immediately.
However, on some days, like the day before Christmas, this would end up in extreme disappointment and a lot of time wasted. You probably would have no idea why you’re waiting for so long. Is it because the queue is too long? Is a single bad job blocking the teller? How many tellers are there?
The example above, though simplistic, describes exactly how teams that poorly manage their queues operate. They may observe the effects of the long task queues i.e. longer cycle times and wait times while blinding themselves from the necessary information they need to solve this problem.
Kanban, a method based on queueing theory, recognises this and insists on limiting work in progress in each of the process queues. Limiting Work In Progress shortens the lead and cycle time for features, it also reduces context switching greatly improving productivity. These limits on WIP help us to differentiate between almost finished work and done work. Additionally, it ensures that all work that enters the system actually leaves the system.
This is critical for getting the benefits of a queueing discipline, because queues don’t last forever, and if work doesn’t leave the system you’re assessing, it isn’t a queueing system.
Simple tools to manage your queues
Reinertsen(2016) gives some ideas on how to manage your queues better
Cumulative Flow Diagrams
While looking for a photo for this section, I came across this super article by Chris Cooney. I couldn’t resist taking a screenshot of his graph and its cheeky caption.
Cumulative Flow Diagrams depict what is happening in a queue over a period of time.
The top line of each band on the cumulative flow chart represents the entry point of tasks in the respective stage of your Kanban board while the bottom one shows when it leaves it. If a line becomes flat, that means nothing is arriving in the corresponding stage or nothing is leaving it.
Using a CFD, you can get an idea of the size of your queues and the cycle time in just a single glance. We plot cumulative arrivals at a process as one variable and cumulative departures as a second variable.
The vertical distance between the arrival line and the departure line is work that has arrived but not yet departed, thus, the instantaneous size of the queue.
The horizontal distance between the arrival and departure line tells us the cycle time through the process for an item of work.
The total area between these two lines tells us the size of the queue.
At a glance, the CFD helps team understand the characterisation of their queues.
Little’s Law
Little’s law is a theorem by John Little which states that the long-term average number L of customers in a stationary system is equal to the long-term average effective arrival rate λ multiplied by the average time W that a customer spends in the system.
Expressed algebraically the law is L=λW
Almost any queuing system and even any sub-system (think about the single teller in a supermarket) can be assessed using the law.
Little’s law allows us to predict cycle time based on the quantity and the processing rate. For example, if our testing area has 50 tests in process and completes an average of 10 tests per week, it will average 5 weeks to complete the test.
As mentioned earlier, a queueing system needs to be one in which work actually leaves.
Having items with an indeterminate wait time leads to an inaccurate work in progress time, therefore Little’s law can’t be applied.
Managing Queues as Opposed to Cycle Time
We have seen how cycle time is a key metric in product development work and the most efficient teams monitor it closely. However, cycle time is a lagging indicator to queue length. Back to our supermarket example:
Consider a bus with 50 university students attending a music festival arrives at the supermarket. The students all pick their items and head to the tills at 9:00 AM. Assuming the queue already had 10 customers, immediately the queue length, increases by 50 arrivals to 6 times its original size. If each customer took an average of 3 minutes, the first customers to experience a doubling of this cycle time to 6 minutes will be, according to Little’s Law, those served at 9:36 AM.
This shows that monitoring queues alerts us to a problem faster, than monitoring cycle time.
Conclusion
The key metrics that agile teams are most concerned about are affected directly by the queue length. Shortening queues is a simple and cheap way to increase agility in your process. Being mindful of our queues leads to more efficient ways of working as well as an increase in quality. The question that remains is, Do you manage your queues?
Reinertsen, Donald G.. The Principles of Product Development Flow: Second Generation Lean Product Development. Celeritas Publishing.
