Why do we use Scrum? About failing as early and quickly as possible
Sometimes, people ask me when they should use Scrum or a more traditional planned approach (like waterfall) for their projects. The reasoning here is usually that a customer does not want to use Scrum, that a project is fixed price, that a project is too small or that people don’t see the benefits of Scrum. Although I certainly understand where these people are coming from, it does tell me that they haven’t truly grasped the core problem with any planned approach to software development: the ever increasing heap of assumptions and the inability to test these assumptions early and frequently.
You know, before we start developing any actual software, we’re making a lot of assumptions about what functionality we’re going to need down the road. We’re assuming we understand the problem and the solution. We’re assuming what end users are looking for and what makes them use it. We’re assuming the time it will take to deliver functionality. We’re assuming what is technically (im)possible. We’re assuming that functionality will generate business value. And we’re also assuming that we are all on the same page on how we interpret functionality. We’re even assuming that our project strategy for developing software is the right one. Every single one of these assumption affects the success of our project. If our assumptions are wrong we are failing.
In planned approaches we are creating ever increasing mountains of assumptions called ‘plans’, like ‘functional designs’, ‘technical designs’, ‘UX plans’, ‘visual designs’, ‘Gantt charts’ and whatnot. No matter the effort invested by skilled engineers, analysts and designers, the resulting documents by themselves provide us with no means of checking those assumptions. The only way to truly verify our assumptions is by creating working software that can be experienced and used by customers, stakeholders and end users.
If our assumptions are always right, a planned approach would work very well. But with a success rate of only 14%, it clearly doesn’t. The reality is that most of our assumptions will turn out to be wrong. Maybe functionality that we believed to be easy turns out to be very difficult and time-consuming. Or maybe our end users are not happy with it, even though we thought they would be. Or maybe we forgot about some essential piece of functionality, but it only becomes obvious when we can testdrive the software. A planned approach makes these failures very expensive because we can only check our assumptions near the very end of the project, when working software is finally delivered. But by then, it is usually too late, too expensive or too difficult to correct for wrong assumptions. A failed, or at least an unsuccessful project, is the result.
So, why do we use Scrum? Simple. Scrum allows us to quickly and frequently check our assumptions in order to prevent expensive failures. Scrum does not magically help us avoid wrong assumptions, but it helps us to check them as quickly and early as possible by delivering working software every sprint. Because the sooner you know that your assumptions are wrong, the more options you have. You can either drop the functionality, ignore it, change it or extend it.
Scrum provides us with a number of means to identify incorrect assumptions. The sprint review is the most obvious one, as it helps us to verify assumptions about functionality, estimates and business value. The Retrospective helps us identify failure in our process (e.g. by failing the sprint, low velocity, jerky burndown, etc). Maybe we are making wrong assumptions about working together, communication or Scrum itself. Even the Daily Scrum helps us check assumptions, as every session provides us with an opportunity to inspect progress of our team and adjust our strategy if things are not going well.
The bottom line is that Scrum is not going to magically make all projects successful. It is also not going to prevent incorrect assumptions and the resulting failures. Scrum will help you prevent costly mistakes as a result of wrong assumptions, by detecting them very early and frequently. This is always better than approaches that do not provide this opportunity.
So, no, I don’t see any reason why you would ever opt for a planned approach over Scrum. If a customer or a team does not see the value of Scrum, they should be educated on the above. If a project is small (a few weeks), even small — but wrong — assumptions will have a massive impact on a small budget. If a project is fixed price, it is even more important to avoid wrong assumptions and the resulting costly mistakes.
The only reason why I would not use Scrum for a project is when there is another approach that can more easily (e.g. with less overhead) detect wrong assumptions, quickly and frequently.