The case against Waterfall
I genuinely believe the Agile manifesto outlines a great set of principles for a successful development team. As a certified SCRUM practitioner, I also believe that formalized project management techniques are helping large teams to improve their throughput.
Every successful team has to find a unique way of shipping great products. Unfortunately, some patterns are preventing teams from defining their unique way, and I would like to share some of my observations of these.
Before Agile
No sane person advocates for Waterfall. Repeat, there are zero sane Waterfall advocates. That said, every argument that Agile adherents make against their opponents could easily be made by those imaginary Waterfall adherents they fear so much with a slight change in wording. @ayasin
When joining a new team, I am trying to understand the product legacy. People are saying, that teams have switched to Agile from…say, Waterfall. So I ask, why that happened. This is followed by classic reasoning. BDUF attempts never worked for us, it took too long before a single line of code was written, detailed specifications documents were never accurate, every time the team developed a product increment, they only realized that it is not what users or stakeholders wanted and so on.
This assumes the existence of a mythical Big-Design-Up-Front detailed documentation, that was executed but the resulting product failed to succeed.
So I ask — can I see a sample of this documentation? Was it a Word doc? Can I see your old Backlog or specifications?
And you might guess my point — this mythical document almost never exists. But how can you know, that upfront design practice was the root cause of your problems if you never executed it?
Waterfall is bad but it is not the problem
During the early days with a new product team, I met with several company veterans, that were not exactly passionate about the product objectives. It was arguably solving the single most pressing issue of the existing user base, so I was interested in hearing their feedback.
I found out that the reservations came from the fact that the company spec’ed the same product feature a few years back, but it keeps failing to ship it to the market ever since, and they thought the feature lacked business support, not product vision.
I keep designing in iterative Scrum with research and prototype validations while digging for the BDUF in parallel. A few months later, I found it — a mythical Word Document! It was not fancy in its form. Still, it contained product goals and user benefits, UML diagrams of user interaction, and a clear set of functional specs with reliable technical feasibility validation and gap analysis outlining a staged and iterative development delivery plan. The document defined pretty much the product that I was working on for the next year. I was often referring back to the initial doc when in trouble and going through trenches of delivering the product within the perfectly defined Agile development practice.
Was it a Big-Design-Upfront problem that this organization needed to solve?
Goals and Objectives
It was only once in a lifetime experience that I witnessed the existence of BDUF product documentation. Its non-existence often hints to the fact that teams operated with no Backlog at all. Not only they miss any form of tangible product specifications defined before writing code of it. They were unsure about who is doing what and what are their roles and responsibilities.
Agile and SCRUM came in as a silver bullet not only to define the delivery model but provide necessary guidance about the roles.
It is a significant shift, but not enough for the challenges beyond the product features. Teams are often struggling with management chaos. No sane and responsible executive should allow the team to work on throw-away work or product that nobody wanted, and nobody pays for — also-known as Waterfall. Yet we see only the delivery teams go through the shifts and movements of implementing new tools, techniques, and methods such as Agile and SCRUM.
The management roles on all levels are defining incentives, responsibilities, and reporting models. They have a long way to move from the industrial era and are far away from leading the innovations.
With an expectation of failure, it is no wonder that managers have little confidence in the outcome of software projects. They’ve seen giant, successful, well-funded companies spend years launching colossal failures, like Windows Vista and Yahoo’s Panama. They’ve also seen undergrads and misfits with hand-me-down computers create colossal successes, like FaceBook and Flickr. Well, if you don’t understand software, that can seem pretty random. And if you think success is random, then spending less for a lottery ticket makes more sense than spending more. It is this lack of confidence in the result, along with the knowledge that the size of the investment doesn’t correlate to success that makes managers wary of spending money on software…Alan Cooper
