Hurry up, For the Sprint Deadline is Approaching!
In a previous post I alluded to a dislike of fixed-sized sprints. I said that I would cover why in another article, which is what this post is about. But let’s start with what exactly a Sprint is. From the Scrum Guide (p9):
The heart of Scrum is a Sprint, a time-box of one month or less during which a “Done”, useable, and potentially releasable product Increment is created. Sprints have consistent durations throughout a development effort. A new Sprint starts immediately after the conclusion of the previous Sprint.
But having worked with fixed-length Sprints for a while, I’ve come to find them rigid and at times difficult to work with. Let’s discuss why…
Packing Work Into Fixed Size Time-Boxes
If Sprints are a consistent, fixed-size duration, and if work items (or stories) are of varying size or duration to complete, then assigning work to a Sprint could be described as a “bin packing problem”. That is, we have items of varying sizes (the stories) and we want to pack them into containers of a fixed size (the Sprints) with a minimum number of containers used (fewest Sprints to completion). We use a bin-packing algorithm (the Sprint Planning session) to pack each bin in the queue.
On the surface this seems to make sense. So where’s the problem?
The first problem is that in the traditional bin packing problem, we accept that some bins will still have some empty space within them where we couldn’t find an item to fit. We simply close that bin’s lid and move onto the next bin. But in a typical Sprint planning we can’t have empty space in the Sprint — that would be idle time for our development team! So in an effort to use up every last little bit of space in the Sprint we find another story and pack it in also. If we think a story might be a little too large to fit correctly, we add it in anyway. That is, we overstuff the Sprint with work.
The second problem is that stories sizes are estimated. Their actual size can vary substantially from the estimate. Something that was estimated as two days might actually take four days to complete. But our Sprint boundary cannot flex to accommodate this time difference. Due to the first problem above we rarely have spare capacity in the Sprint, so we end up with both overstuffed and expanding work in the Sprint at the same time.
The third problem is that when we accept work into a Sprint, there is an implied agreement that we will complete it. That is, the sprint planning session itself transmutes an estimate into a commitment. So you end up with overfull Sprints of work where each item has the potential to explode in volume while that Sprint is underway, but an expectation by the business that it “absolutely, positively” must get done by the Sprint deadline.
I don’t know about you, but to me this seems like a recipe for potential disaster. I’ve seen all three of the above problems manifest, sometimes all at the same time!
A Sprint Boundary is an Immovable, Artificial Deadline
Despite its intent, that’s ultimately all that a Sprint boundary is — “by this fixed date your team will deliver a working product with all these stories that you’ve committed to complete.”
But two questions need to be asked. First, why do we actually need a fixed and immovable deadline? Second, how did a story estimate transmute into a firm commitment?
I’ve come to the view that Sprints are a consistent fixed length for 3 reasons:
- To provide a common baseline for measuring velocity. And yet the original purpose of tracking velocity is to estimate how much work we can draw into a Sprint. It’s a circular system. By “measuring” how much work we can draw into a Sprint, theoretically, we are supposed to have more confidence that we can deliver it in the allocated time.
- To provide an external commitment to other departments for fixed delivery dates.
- As a form of discipline, to prevent “drifting” into longer and longer time frames for each Sprint as work progresses, i.e. to ensure regular delivery.
Because we are using metrics that look sophisticated from the outside, and because they are intended to give us confidence in our calculations, people will naturally come to see them as voluntary commitments. This is especially true for people in management, other departments, or parts of the organization that don’t understand how software development works — they rely on us as experts to make decisions and trust us to stick to those commitments. (And don’t forget that they are under their own pressures too, which are foremost in their minds.) It’s just like asking a contractor for a written quotation — you expect them to complete the work specified for that fixed price. And for software delivery, from the outside, it’s the same. Our semi-sophisticated but error-prone methods of estimation using incomplete information at the time of planning lead to expectations that we will actually do what we said we would do within the budget and time that we agreed to.
If we do not deliver as per expectations, questions are often asked as to why not? Typically this pressure comes from stakeholders, management or the business, who all speak daily in terms of commitments to one another, and who implicitly trusted us to get on with the job and complete it. In the best case, we squirm as we are required to explain ourselves like naughty little children who spent too much time playing and not enough time working. In the worst case, we get an “unreliable” black mark against our name. It’s happened to me more than once.
So by the time a Sprint starts we’ve accepted (often too much) work into it despite being unsure how long it will take to complete. But the Sprint is fixed-length and there is a fixed expectation that we will deliver it. To me, this just sounds like setting your team up for potential failure!
The Politics of Sprints
Remember how we said that Sprints could be thought of like bin-packing, only it doesn’t quite work correctly? Well, from an outside management point of view, it’s often looked at exactly as a bin packing problem. And this brings a raft of political problems to the teams and to the wider organization:
- The transmutation from “estimate” to “commitment” creates expectations throughout the business that we will deliver this work by the end of this Sprint. If we fail to deliver then we have failed to live up to our commitment. Most companies don’t have a culture that permits frequent and regular failure, especially failure to deliver a piece of work that we (apparently) voluntarily “committed” to. So there is immense pressure to deliver something, even if it’s cut in scope or rushed or both.
- Sprint boundaries create a problem of waiting. If we cannot deliver a “promised” feature this Sprint and need to push it into the next Sprint for completion, then the business will have to wait another whole Sprint to get that missing feature! So under pressure to deliver “something” sooner we end up cutting the feature to force-fit a delivery into the current Sprint (see the first point above).
- If we’re doing SAFe or any other variant of Scrum where a large block of work is assigned to a block of several Sprints in advance (known as a Program Increment or PI), then we have much less ability to rollover overflow work to the next Sprint. This is because the subsequent block of Sprints already is packed full with more pre-assigned work. That is, there’s no spare capacity to rollover work. Any unfinished features from the current Sprint or PI need to wait to for a future planning session far in the future to be re-assigned for completion, which further magnifies the second point above and creates more a sense of urgency to finish the feature now. As in, “if you don’t finish the feature now it will never get finished, so you have to deliver something now, even if it’s not ideal.”
In my experience the above leads to a lot of pressure to ensure that we deliver every allocated feature by the end of the assigned Sprint. And that leads to…
Hurry up, For the Sprint Deadline is Approaching!
As the Sprint deadline looms closer, teams experiencing the above problems get squeezed for time, and may reduce themselves to do things like:
- Cut scope. This risks removing some of the more important, common or profitable options from the feature.
- Cut corners. Also known as rushing the work. This will often result in accelerating technical debt as we hack code together in the quickest way possible. In other words, quality takes time.
- Drop product discovery work, refactoring work, architecture work, and any other work that doesn’t directly contribute to velocity points (more on velocity later). Mike Cohn makes the point that Agile teams still do emergent planning and emergent architecture, but in my experience this rarely happens when under pressure.
- Work too much overtime consistently, resulting in the risk of burnout.
Under mounting pressure, the name “Sprint” becomes very apt. Because we feel like we have to literally sprint to the finish line to make good on all those promises of work that we would deliver. And then the next Sprint starts, and we repeat the pressure cooker scenario all over again!
What Would Happen if We Extended a Sprint Deadline?
Clearly, as per the section “The Politics of Sprints” above, we don’t want to miss a Sprint deadline entirely. But what would happen if we were a day late, or even two days late? Would the world end? Would our managers go crazy? Would our product die?
Except for some rare circumstances (e.g. we need vote tallying to go live on a fixed election date) most of the time nothing bad that matters will actually happen — there will not be any dire consequences for our business. For the most part, the only bad things that might happen are internal to our organization and largely imagined. For example, we might get a reputation that we missed the Sprint deadline by a whole two days! Shocking, I know!
So why do we tie ourselves to a fixed-length Sprint deadline when there isn’t actually any catastrophic material fallout from not doing so?
What Would Happen if Sprint Boundaries Were Flexible?
Here’s a thought experiment: if nothing materially bad would actually happen if we missed a Sprint deadline by a few days, what would happen if we allowed Sprints to be flexible in duration rather than fixed?
My first instinct is to say that due to the optimistic bin packing algorithm that is Sprint planning, we would be a few days late every Sprint. Our Sprint boundaries — which are arbitrary dates on a calendar for the most part — would be off every time. Remember our discussion about expectations above? Those dates mark fixed and concrete expectations. If we keep missing those dates then we are likely to cause disappointment and frustration to people in other business units who are depending on them.
In the worst case, each time our Sprints would become longer and longer, and eventually we may have devolved to the point that a Sprint is two to three times longer than the initial Sprint simply because we accrete a little more lateness to each Sprint every time. We couldn’t measure velocity if our Sprints were all of different duration.
What You Measure Eventually Becomes The Only Thing That Matters
If you only measure velocity, then ultimately only velocity will matter. Any other work that does not directly contribute to velocity will ultimately get cut. This includes product discovery, and architecture, and planning. It takes time to create an architecture. It takes even more time to build to an architecture, when a sticky-tape and bubble-gum implementation can be done much faster and still deliver “business value”. Jeff Gothelf is right when he suggests that velocity could be renamed to “future tech debt”.
It seems to me that there are ultimately two reasons that velocity exists:
- To provide some bin-packing metric for fixed-sized Sprints.
- To give management a simple number that they can “optimize for productivity”.
If we could provide an alternative to the bin-packing approach to Sprints then we could do away with velocity for number 1. My previous post discusses this in much greater detail.
If we could get management and the business to focus more on measuring “outcome” (i.e. the actual customer impact of our product) rather than “output” (i.e. delivery velocity) then we would not need velocity for number 2. Both Jeff’s post and my previous post discuss this is some detail, but optimizing for customer outcome rather than optimizing for code output is a current trend in effective product management.
If we could achieve both of those, we wouldn’t need to measure velocity, and thus we wouldn’t need Sprints to be fixed-size just to act as a baseline for that measurement. But we still need to deal with that problem of varying sized Sprints for setting external expectations.
Why Even Have Sprint Boundaries At All?
Imagine if there was a version of Scrum that did away with Sprints and Sprint boundaries entirely. It’s called Kanban.
Instead of limiting work by total volume per Sprint (whose problems I detailed in my previous post), we instead limit work by limiting the number of items currently in progress. That is, our measure for ensuring that work gets completed in a timely manner is that we must not commence something new until the current work item is finished and off the Kanban board.
Instead of setting external expectations of when work will be finished as a bunch of features all delivered on a joint date (the Sprint boundary), we could instead set expectations for each feature individually. This would allow us to stagger the delivery of features to deliver continuously rather than all in one bundle. Moreover it allows us to fine-tune and adjust an individual feature together with the business without the pressures of conforming to a Sprint boundary.
What I like about Kanban is that it captures most of the advantages of incremental delivery with very few of the disadvantages that Sprints introduce. Of course Kanban itself does not specifically include other useful ceremonies from Scrum but there is nothing stopping us from doing those things at a regular cadence. For example, we can still hold the equivalent of a Sprint Review at a regular cadence, it would just be more like “throughout the past two weeks, these are all the things that we delivered.” The only thing we have to be wary of having the discipline to keep those practices up. In my opinion, the main thing that Scrum provides out of the box is enforcing discipline of the four ceremonies for us.
Conclusion
Incremental delivery doesn’t need to mean fixed-length increments. We can still deliver incrementally if we allow some flexibility in our deadlines. In fact, the rigidity of Sprint timelines could be considered to be the opposite of agility, because we are not adjusting the program to take situational information into account.
Kanban is a lighter-weight way of delivering incrementally. It has less overhead and less ceremony than Scrum. Especially when taken together with my previous article on story points, we can eliminate a lot of overhead. It allows us to focus less of our attention on managing internal processes and more on building great software. It does however require greater discipline of the team to address some of the areas that Sprints cover automatically, such as sticking to a regular cadence and communicating delivery dates to the wider organization. With some practice however, I do not believe this to be difficult. However, in order to see the forest through the trees and ensure that the team is working towards a consistent goal, a coherent and compelling product vision becomes even more important in Kanban than in Scrum. But we needed to do that anyway :)
