Think Outside the Box: The Sad Truth about Agile Timeboxes
Frameworks such as Scrum have codified Agile development in a way that have allowed engineering organizations to move more quickly and with higher productivity than in Waterfall. That said, after observing many hundreds of iterations across Scrum, XP, and SAFe frameworks over the span of 15 years, it is apparent that timeboxed Agile frameworks are an impediment to realizing significant productivity gains through Agile practices. Timeboxes by their nature impose needless toil on a development team and organization in the form of eager planning. Planning is poison to a development team. In small doses, it is sustainable. Unfortunately, timebox frameworks require a lethal dose.
The setup
What is really so bad about planning anyway? To understand this, we need to peel back the onion. Planning in a timebox framework consists of two key activities which I will refer to as dimensioning and fitting. (I am purposefully not using the term “sizing” so as to avoid any subconscious baggage that may come along with the term’s use in popular Agile frameworks.) We will see how the combination of dimensioning and fitting spells doom to our productivity as a product development team. First, let’s define these terms.
Dimensioning is the process of assigning a magnitude of effort to a backlog item. For example, given a user story, a team will need to use some approach to give dimensions to it so that it can later be assessed to be “in” a Sprint or to be seen as not fitting into a Sprint. Having dimensions assigned to backlog items also helps business stakeholders in prioritizing work as they can then see when it is possible to raise the priority of a smaller work item in order to get it done faster.
Fitting is a process which follows dimensioning in timebox framewoks, where the product team juggles items at the top of the backlog so that an acceptable arrangement of them fit into the upcoming timebox, the dimensions of which are prescribed by the software development framework. Many of us are familiar with Sprint planning in Scrum. Fitting occurs during this session, with the output being a group of backlog items (stories, tasks, etc) committed or projected to be completed within the forthcoming timebox.
Now that we have defined the activities, what is so wrong with them?
Dimensioning forces a team to choose between anxiety and misery.
Understanding that the dimensions given to an item of work will be later used as a means for making a commitment to getting a certain batch of work done in a fixed period of time, developers will first want to deeply understand the nature of this work. What follows is an in-depth, thorough decomposition of a requirement into design and tangible tasks. This completely natural tendency to get down to brass tacks stands opposed to management’s desire to keep planning activity to a minimum, as it is seen as overhead and the absence of code being written. Thus a tension is created between development and the business stakeholders or management regarding the resolution of work to be completed in the upcoming timebox. Neither settling for coarsely-defined work nor undertaking fine-grained resolution of tasks is the answer. Both have negative repercussions, and that is because the problem we are trying to solve is itself artificially manufactured and flawed.
Dimensioning leading to anxiety
On the one hand, we see organizations with a strong Agile coach or scrum master who will urge the team to not overthink or get too far into design or implementation details during planning sessions. The planning session may even be itself time-boxed. While this “keeping it shallow” approach may seem sound and even Agile, it cuts against the grain of most engineers. A development team, in such an environment, will leave Sprint planning sessions with anxiety. They will not know what they were just signed up for, and feel as if they have just been bullied into committing to a goal they do not understand how they will achieve. Having personally been in both roles (the engineer and the one enforcing the time-boxing), experience has born out that this will backfire and, over time, it will erode trust and any sense of accomplishment and productivity that exists within the team. Achieving the objective and sense of accomplishment is critically important to the health of any development team. To this end, a team will either overwork in order to meet it, or they will fall short and have to report a Sprint incomplete (very commonly the case). Either of these outcomes further compound the decline in team morale.
Over time, teams may resort to sandbagging, or grossly inflating the dimensions of work items, as a means of insulating themselves from these effects. This coping mechanism, however, comes with it’s own hit to productivity, in the form of chronic underachievement, a self-fulfilling prophecy in which the team begins to believe they can only get as much done as the inflated dimensions suggest. For more on this, see Parkinson’s Law.
Dimensioning leading to misery
The alternate way organizations deal with the planning tension is to provide the development team all the time they require to completely break down stories into fine-grained implementation details. While this does alleviate the anxiety resulting from the first approach, it is an excessively expensive activity and, in the end, will still result in frustration, just of a different kind. In open-ended planning, we ask the team to think ahead for the next three weeks (or Sprint length) and often beyond that so as to anticipate downstream dependencies and design consequences. A detailed breakdown like this can extend a 1 or 2 hour planning session into multiple days. Not only is this exercise a costly time commitment during the Sprint planning phase itself, but will continue to impact the team because, in many cases, the work is throw-away and will need to be done over again. Why? The body of work planned for a Sprint is rarely completed in full (an undeniable observation from the field). As this occurs, the unfinished work is placed back on the backlog and waste is accrued in the form of more estimation work, with the carried-over work being taken back up in a future Sprint. These same tasks that were decomposed in painstaking detail during the original Sprint must be inspected once again with equal rigor in the subsequent Sprint.
This option for handling the tension during planning is just as likely as the first to grate on a development team. They are stuck in a loop of estimating the same work over and over again instead of making contiguous progress on it and completing it. Team focus becomes progressively fragmented. Misery sets in as the team sees their time being chronically wasted.
Don’t make the choice. Stop planning.
We need not pick our poison, taking either anxiety or misery. There is another option. Don’t plan. By this we do not mean that estimating work is never appropriate or necessary. It is the type of dimensioning of work that is required of timeboxed frameworks that is the problem. Timeboxes are boxes. Boxes hold batches of things. If we have to dimension a batch of items, by definition the majority of these items are being dimensioned in advance of the item actually being worked. This lead time between thought and labor is the root of the poison flower. Information given at time of the original detailed analysis ages — priorities change, architectures shift, technologies mature, requirements themselves evolve.
Timeboxes exacerbate this reality by prescribing lead time, typically significant lead time. A 1 month Sprint is worse than a 3 week Sprint, which is in turn worse than a 1 week Sprint (most shops are multi-week or more). The more compressed we can make the time between thinking deeply about something and doing the thing, the better.
Fitting
Now that we have painstakingly dimensioned a batch of items, it is time to apply the timebox and dismember them. What?? In planning sessions, our duty is to fill the timebox as completely as possible, and thereby commit to getting a batch of work done by the time the timebox has expired. The box is a rigid one. If a work item doesn’t fit nicely into it, it is the task cohesion that is sacrificed, not the structural integrity of the box.
Timeboxes are the Procrustean beds of software development.
Here it is helpful to turn to Greek Mythology for an analogy. Procrustes is a malevolent mythological character who forced his victims to fit his iron bed. He was compelled by some dark force to make them fit this bed precisely; by brute force. If the victim was too short for the bed, he would literally stretch them to fit. If too tall, he would hack off limbs.
Timeboxes are the Procrustean beds of software development. The process of fitting work into a Sprint is no less of an ugly and brutal exercise, often forcing or smashing work into odd shapes or fissuring work in unnatural ways.
Agile practitioners will certainly be able to recall situations during Sprint planning sessions when teams had to break down a large user story that didn’t fit neatly into the remaining available volume of a Sprint. There are various techniques to employ in doing this. The story may be broken into multiple composite stories along themes or milestones, or perhaps a “spike” is extruded from the original story, which as no coincidence, happens to be the exact dimensions of the remaining volume of the box. Integrity of the original work item is lost, no matter what method is followed.
To illustrate the point, let us consider a very simple example where there is a user story: “As a retail consumer, I want to be able to apply for credit in order to purchase items over $1k in value”. Let us also assume the team has dimensioned this story to have a magnitude of “8 units” (what a unit means is not relevant). During planning, the team assesses the available volume to fill at 25 units based on historical velocity calculations. Business stakeholders will expect their 25 units to be spent completely. They do not want to be shortchanged. The team has already discussed 4 other work items (that total 22 units) to be committed in this Sprint. 3 units are left to be spent. With this credit purchase story as the next highest priority item, a discussion ensues. A business stakeholder is likely to ask “What progress can we make on the credit purchase story in this Sprint? I’ve been waiting for it for a long time. We have to move this along. I can’t wait for the next one.” Inevitably, the development team makes an attempt at splitting the story. Out may come a spike to research or to build a library to support the feature. Another option is the “phased” approach, where the story is duplicated into “Credit Purchase — Phase 1” and “Credit Purchase — Phase 2”, where the phase 1 story happens to be of magnitude.. you guessed it.. 3! We start to see here a Russian doll syndrome, where box begets box begets box. What is a phase 1 or a phase 2 other than just another box. It is very easy to lose our bearings on what is the desired interaction or feature we are ultimately providing.
These are examples of hacking an otherwise cohesive, intelligible unit of work into arbitrary pieces, so that it fits the iron bed of the timebox or Sprint.
Lest we think this is the extent of the damage to be inflicted, there is the stretching kind as well. Recall earlier how we mentioned Parkinson’s Law. Timeboxes create fertile ground for estimation inflation to set in, where, like a gas expanding to fill a space, teams will inflate the dimensions of work items to fill the volume of the container. The arbitrarily sized container can indeed be filled very neatly by arbitrarily sized items, designed to fit it. Can we recognize the futility? We have fashioned a game and have “gamed” our game. Let us congratulate ourselves. We must consider what this is doing to our teams’ sense of accomplishment, velocities, morale, and our mental conditioning about the urgency and importance of work.
Timeboxes don’t align with rest of the business
As a final point, consider how the software industry continues to pivot to releasing in smaller and smaller increments. The gap, between the way that we plan or execute work and how we deliver the finished product into production, is widening.
Businesses want to talk features, but we have trained them that this is not allowed.
Product management prefers to think in terms of backlogs of items, ranked lists of discrete and atomic blocks of work that provide business value. PMs care about their feature moving from their backlog into development, and then into production and providing value to the customer. Notice there is no mention of timeboxes or batches of work. There are only features.
On the release side, continuous deployment or continuous delivery (CD) is likewise allowing us to think in terms of smaller, more quantized packets of value reaching the customer — features, as opposed to releases. When a feature is finished, it is shipped. Batches of features are viewed negatively. Batches represent risk due to a greater delta of changes being introduced to production at a given time. Much focus is placed on eliminating anything that will get in the way of a single feature providing value to a customer as quickly as possible.
In the middle of these two activities lies development. Leading into it we have discrete features, and emerging from it, we again have features, but inside it we have Sprints or iterations or increments. Timebox frameworks force us to think in terms of batches. The best we can say is that among a group of possibly unrelated work, a sought after feature will be potentially shippable at the end of a Sprint. Businesses want to talk features, but we have trained them that this is not allowed. As a result, many have lost patience with Agile and are frustrated by the opacity, lossy translation, and extra needless effort that seems to come with it. They want visibility into features. They care nothing about timeboxes.
In Conclusion
Timeboxes set us up for failure, underperformance, and burnout. The good news is, we don’t have to use them! We can escape the illusion of better outcome by rigorous estimation and planning, and we can break free of the arbitrary timeboxes. This is possible by applying Lean and Kanban principles, beginning to work in a piece-wise manner throughout our entire development processes. Timeboxes evaporate and what remains is a backlog of naturally shaped and just-in-time dimensioned work items. These backlog items flow through development processes completely in tact and as part of no batch. In a subsequent post we will discuss how to operate in this new world without timeboxes and without planning sessions. For now we will leave with this thought: by being more continuous throughout the entire development process we can avoid the poison of timeboxes and planning and instead more naturally mesh with all the surrounding gears of the business, being no less Agile whatsoever.
