Is Agile just a bunch of BS?

“I’m just gonna say it. I think Agile is bullshit.”

I think I may have flinched visibly when I heard those words from my friend, a seasoned UX professional whose opinion I respect. I was surprised, to say the least, and I think she saw it on my face. I had just finished reading “Sense and Respond”, and I was full of renewed excitement about Lean and Agile approaches to product design. But I was curious about what she meant, and went into consultant mode. “Why do you say that?” I replied.

“It doesn’t work. I’ve seen countless companies try it, and it never works the way it’s supposed to. I mean, they all SAY they’re being Agile. But in the end it’s just the same old thing, just dressed up with sprints and backlogs. If 80% of all instances of Agile implementation are dysfunctional, it all can’t be because ‘They aren’t implementing it correctly.’ Something bigger has got to be wrong.”

I started to object, but she pressed me. “Come on. Have you ever actually SEEN a company with a good, functioning Agile process?”

I had to think about that. I had met people at Meetups and the like, who used what sounded like working Agile processes; but I’d never actually worked with them. Certainly my own clients — mostly large companies — weren’t doing anything that I would really call Agile. (My own consulting team used an Agile-ish process, but wedged into a fixed-price contract, which I’ll admit takes away a lot of the agility.) But surely the problem was that these were missing the point of it, not implementing it well…?

“You could say it’s just not being done right, but at some point, if so many people keep trying it and it still doesn’t work, maybe the problem is the approach itself — not the implementation.”

It was a good point, and I had to sit with that for a while. In particular, it led me to ask myself: What would it mean to succeed or fail at Agile? What would a successful Agile adoption look like?

And what was supposed to be so great about Agile in the first place?

The Agile movement arose as a result of abusive practices in the industry. It was a response to product managers piling on requirements and giving developers fixed deadlines by which to achieve them. It was a response to management dictating solutions, and changing requirements in the middle of development. It was a response to huge projects, planned in elaborate detail, that resulted in products which weren’t needed.

There was a lot of buzz about this new approach, which was outlined in the Agile Manifesto. We heard that that Agile teams delivered products faster, and were happier and less stressed. What’s more, this approach promised to enable companies to respond more quickly to changes in the market. Naturally, lots of people were excited about this, from developers to business executives.

When I went through my first Agile adoption, the company I worked for gave us training in Scrum — as well as Agile product management guidance from Marty Cagan, whose book Inspired is still one of my favorite books on product management. I was intrigued; and in general my colleagues seemed excited about it, and eager to try it out.

And as for so many teams, the reality was a huge disappointment. Design teams rushed to “feed the beast”, pushing half-baked designs to development teams which were accustomed to fully-thought-through specs. Management constantly pushed the teams to increase their velocity. QA team members struggled to get stories approved before the end of the sprint. The director of product management continued dictating changes from above, regardless of the timing of the sprint cycle, and overruling the teams’ “product owners”. And management continued their old practice of releases with fixed feature sets AND fixed dates, forcing teams to take on more work each sprint than they believed they should. No one on the team was happy with the change.

Experienced Agile practitioners will see plenty of mistakes there. We saw them as well; and we raised them in retrospective after retrospective meeting — but nothing changed. Still, one could argue that this is an issue of bad management, not a failure of the methodology itself. And it’s true, we definitely had bad management, at least in the product management and development organizations. I don’t mean that they were merely pushing us too hard, or lacking in leadership; rather, they were directly violating the principles we had just learned together. They were clearly “doing it wrong”.

I think that recognizing this fact was a big part of why I didn’t turn against Agile in general. With all the flaws of our company’s implementation, there were still clear advantages to this new way of doing things. The close collaboration between development, design and QA was a breath of fresh air. The cyclical nature of the work lent itself well to user testing, which always ended up being dropped from ‘waterfall’ projects when schedules or budgets slipped. And the ability to push back on management — to say “that’s a great idea, Mike, put it on the backlog and we’ll get to it next sprint” — definitely felt like a step in the right direction.

Since then, I’ve worked with many other teams making various claims to being Agile. And I’ve seen a lot of companies with the trappings of Agile, but without the characteristics that Agile was supposed to bring us: the shorter delivery cycles, the faster response to changes in the market — not to mention the happy empowered development teams.

Unlike in my first example, many of these teams weren’t always “doing it wrong” in any obvious way; but something still felt “un-Agile” about their approach. I’ve seen teams work on projects for years without actually releasing software to customers. I’ve seen products with requirements defined in advance, where managers insisted on “finishing” one full feature set at a time, and never returning to it. And I’ve seen jumbled, unusable products resulting from teams that jumped right into development without a good sense of what they were building.

One could argue that each of these examples weren’t “wrong”. The software was still built; each iteration produced working software that stakeholders, at least, could react to; and there was at least the potential to change directions and re-prioritize, to deviate from the plan when circumstances changed. Some people in each of those companies would say that their brand of Agile was working just fine.

And yet something is clearly missing here. There’s good reason for the eye-rolling that inevitably follows when managers trot out “Agile” along with the other buzzwords of the moment: “transformation”, “disruption”, “innovation”, “intrapreneurship”, etc. — a feeling of “same shit, new jargon”. If you’ve been around for a while, you’ve probably seen plenty of cases where companies made a big deal about going Agile, or Lean, and yet things stayed essentially the same. (And I realize I’m partially conflating these two approaches; but they share a lot in common, and I’ve seen similar issues with companies that claim to be adopting Lean.)

So it’s clearly true that “adopting Agile” is not enough to bring those benefits.

But is it the wrong approach?

Take a look back at those values, those principles. They don’t say anything about stand-up meetings every morning, or backlog grooming or sprint planning meetings. They don’t say how complete your designs should be before developers start work on them. They don’t say whether to use Scrum or Kanban or XP or whatever. You could throw out those systems, make up your own, and still follow those principles.

And when I look at those principles, I can’t find anything I disagree with. Sure, there are a few products you wouldn’t want to build in that manner; but if your goal is an organization that is more adaptable, more responsive to change, those principles are still valid.

The real question then becomes: Is the WHOLE organization applying those principles? Or is it just the development or product side?

In all of the cases I’ve mentioned, the company was acting as if this transformation affected only one layer of the organization. Instead of continuing the transformation up the ladder, they just maintained the same old management practices on top of a re-organized product team. It’s rather like a person making a New Year’s resolution to lose weight, and exercising at the gym three times a week, while continuing to snack on Twinkies and fries and watch TV for hours every day. It’s certainly true that exercise can help you lose weight; but to really get the benefits, you need to make larger changes in your life.

And for organizations, as for people, those larger changes are difficult to make — and the temptation to fall back into old habits is very strong.

Ultimately, I do essentially agree with my friend, in that Agile alone won’t change things. But I still think it’s the right approach. I just think we need to go even further in that direction. If an organization is going to reap those larger benefits of Agile — the increased responsiveness, the shorter product cycle, happier employees — they need to challenge a lot of the assumptions they make around how a business is managed. They can’t just ‘go Agile’ at the bottom, while still taking a top-down, command-and-control approach to the other layers of management. The solution, in fact, lies in making a similar transformation throughout the rest of the organization.

There’s no definite roadmap for this, of course, and every organization has its own challenges. But I’m starting to see some great ideas emerge, illuminating a path towards a better way of doing things. “Sense and Respond”, the book which triggered this discussion, outlines some exciting approaches which build on the ideas of Lean Startup and Lean UX. I’ve had some inspiring conversations with people in the Responsive Organizations and Balanced Team movements, and I’ve recently discovered the “Beyond Budgeting” model as well. All of these start from different viewpoints, but ultimately they point in similar directions. Together, these are starting to go beyond the team-based principles of Agile, to map out new approaches to organizing people and projects which could help us achieve the possibilities that the Agile movement started to reveal.

It’d be terribly idealistic to think that any of these will be a panacea, or that our organizations won’t face a lot of difficulties as they try to adopt these new practices. But it’s clear that the old ways aren’t working, as companies struggle to adapt to ever-faster changes in the world around them. And I’m willing to bet that, as more people learn to work in these ways — to approach products as experiments, to welcome user feedback, to trust employees to work in more autonomous teams — we’ll see a change. It won’t be just the slower organizations being “disrupted” or dying off: I predict we’ll see a gradual cultural shift within even the largest organizations, as more people experience the power of these approaches, lowering the resistance to change from within. And I think we’ll see more and more talented people refusing to work at companies that don’t adapt.

Radical new ideas eventually seem obvious after enough people adopt them. I’m seeing it happen already with Agile: some of the younger developers on my teams have never done “waterfall” projects, and don’t see why someone would even consider trying to plan something out in exacting detail before they start building. I believe we’ll see a similar transformation in the way we think of managing and structuring organizations. It may take a while, and some organizations will lag behind others; but I think more and more organizations will achieve the promised benefits of Agile, as these new ideas start to look as obvious to us as Agile does to my younger colleagues now.