“Agile” Is Broken

Alan Kay once compared the “Web” to a pop culture:

“Pop culture lives in the present…People who live in the present often wind up exploiting the present to an extent that it starts removing the possibility of having a future.” [ref: Beyond The Box, February 1, 2007, By Allan E. Alter]

For this reason, Agile software development — as practiced today — is horribly broken.

This flies in the face of the hype, that Agile is the answer for the abysmal history of software projects that end up way beyond schedule and hopelessly over budget. Yet, Agile does not always help: the procurement for the healthcare.gov website specified that Agile should be used, and the contractors claimed — and thought — that they were complying.

It is almost a joke in the Agile community that if Agile is not working for you, then you must “not be doing it right” — that is, Agile can never be to blame.

But what is Agile? It depends who you ask. Most proponents of Agile will point to agilemanifesto.org — a website created by a group of experienced IT folks who were fed up with how software was being approached, and who wrote down a set of ideas to set things on a better path. It was — is — a brilliant document. But as it turns out, very few Agile projects actually follow the Agile Manifesto. Instead, they adhere to specific methodologies such as “Scrum” — assuming that such approaches must be Agile since they are sold as such — yes, “sold”, because Scrum is a commercially marketed process.

As a result, not too many practitioners in the Agile community actually question what they are doing — but instead blindly follow what Scrum says, as well as following the advice of a handful of other thought leaders in the Agile community — few of whom will challenge Scrum since it is so entrenched. In fact, there are a whole suite of entrenched ideas that cannot be challenged without peril — yet, most of these ideas are very destructive if implemented in their extreme form, which is often the case, because as Alan Kay has said, IT is a pop culture, and pop cultures often subscribe to the idea that more is better — whatever the latest fad is. Judgment goes out the window. Thinking is not a factor in pop culture — following the crowd is.

Let’s look at some of the sacred ideas in the Agile community and see how they hold up. Mind you — none of these are stated by the Agile Manifesto, yet these ideas are as much part of today’s Agile as the Manifesto is, because they are so embedded in the culture.

Everyone should be together in the same room

This originates from the Manifesto’s statement that,

“The most efficient and effective method of conveying information to and within a development team is face-to-face conversation.”

Yet, the Agile community has taken this to an extreme, and today an “Agile team room” is considered to be inextricably linked with Agile, so that if you are Agile then you must have an Agile team room — with everyone all sitting together and no walls.

How well does this work? Try this: search the Web for “open plan office” and count the number of articles that push back on this idea. At this moment, the book Deep Work by Cal Newport, is on the New York Times bestseller list. In that book Newport explains why sitting together — compulsory collaboration — keeps people from being able to think deeply. Thinking deeply is necessary for breakthrough insights, and so while sitting together might be enjoyable — it is social — and might increase collaboration — which is a good thing — it comes at the potential cost of losing our best ideas. In short, it threatens breakthrough innovation.

Self organization, and “trust the team”

This originates from the Manifesto’s statement,

“Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.”

Somehow, the Agile community has translated this into an extreme, whereby team leadership is seen as hands-off, where the team should always make every decision and originate every idea. The team lead is reduced to a mere facilitator. Is this best?

In Walter Isaacson’s recent book, The Innovators, he paints a very different picture of what kinds of teams tend to work well and produce innovation. The teams that he describes, which include diverse domains from the development of the hydrogen bomb to the leadership of Intel, are teams that are able to complement each other and that are able to engage in dialectic discussion. The right team mix seems to depend itself on the mix. Very often there is a combination of leaders: someone who can relate to those beyond the team, someone who relates to people within the team, and someone who keeps things running well. But in essentially all cases described in the book, leadership entailed deep knowledge of the subject matter. I did not see a single example of a team lead who was a mere facilitator.

Games as a way to teach

Everyone likes games, right? So games must be a great way to engage a room full of people.

Actually, my wife and I recently had some friends over and my wife — who likes games — tried to get everyone to play a game called “Crimes Against Humanity”. I groaned, and so did my friend Kevin. Kevin does not like games, and I don’t either. We were outvoted, but it made me realize that I am OK for not liking games — there are others like me.

Games are a great learning tool for some people, but not for everyone! We are all different. And even for those who enjoy games, it is not always the best tool.

The purpose of a game-based instruction is that it enables the participants to experience a situation in a tangible way — they live it. However, my wife is a behavioral therapist and she explained to me that for experiential learning to be effective, it is crucial that the context be the same as the context in which the new skill will actually be applied. Thus, for example, a game in which someone cooperates with other players to build a tower of cards does not effectively teach someone to cooperate with others in a very different task such as writing software. The reason is that the contexts are so different that people do not automatically translate the experience from one domain to the other. Thus, games can be entirely ineffective if they do not simulate the specific context in which people are expected to later apply the new knowledge.

What works much better is simulation — going through the very motions that they will use, in a simulation of the setting in which they will later be applying the knowledge — the more realistic the simulation, the better.

Yet games have taken hold as a popular tool for teaching people about Agile, and Agile team coaches all nod their heads when hearing about the use of games for this purpose, when instead they should be shaking their heads.

Dot voting

A colleague of mine recently participated in a management planning session with the CEO of a several thousand person health care organization. The session was facilitated. Afterwards, the colleague told me that it was very ineffective — that she would have greatly preferred to have deep discussion about the issues, either one-on-one or in a very small group with the CEO. Instead, the participants voted on issues and discussed them around a table, and then dot voted on strategies — ending up with a list of strategies that, my colleague claims — is disconnected from the reasoning that went into reaching each strategy.

I have been in many sessions like that, and have always felt the same way afterwards; yet this form of “collaboration” is accepted as a norm in the Agile community. What seems to occur much less frequently is the old form of collaboration, where people discuss things in a logical manner around a table, perhaps with a knowledgeable team lead provoking questions, offering ideas of his or her own, and helping the team to talk things through and reach consensus. That was the approach that Socrates used, and I have always found it to work really well — much better than dot voting, which seems to replace logic with popularity.

An alternative idea

Agile is indeed broken because people are “doing it wrong”, but things have gone so far — there are so many entrenched practices that are simply wrong — that I find that Agile is no longer a useful paradigm for success — we have strayed too far from the original ideas in the Agile Manifesto, by having embraced so many extreme practices that defy common sense. The Agile Manifesto was about balance — not extremes.

Instead, I offer an alternative idea: that we go back to using our brains — which was the intent of Agile in the beginning — and strive to think holistically — consider the whole picture before proposing any strategy or practice; that we never pull a practice out of our pocket and apply it automatically or mindlessly — that every practice be the result of consideration of the whole — the organization, the individuals, the challenges that are faced, the short term and long term goals, and the resources at our disposal. Never assume that things should work a certain way because that is how everyone else is doing it. Shun the comfort of the norm, because, as Michael Tiemann (former CTO of Red Hat) recently pointed out, there is no “normal” and those who do what everyone is doing are bound to be doing the wrong thing.