Agile’s Ethical Dilemma, Decision Distribution, and the Trojan War

Why don’t Agile transformations tend to succeed? It may boil down to a clash between subcultures. Let’s borrow an analogy. Jess McMullin has suggested that Service Design is a sort of “Trojan Horse” for transformation, that to improve the “service” offered you must redesign the org itself. How is Agile different?

The story of the Trojan Horse is a story about war; and, in certain respects, it offers an apt metaphor. In many organizations, a battle of sorts is being fought, even if largely unstated. In this version of the story though, the Greeks don’t offer the gift and pretend to sail away. Here, the Greeks work for Trojan leadership.

The Trojans pay consultants to help build the horse. The Greeks are happy to help, but for different reasons. The Trojans just want a faster horse. They think it’s about transforming the Greeks. The Greeks think something different hides inside, that the horse is about transforming the Trojans.

In this conflict, the “Greeks” are probably more in the right, and that’s where the ethical dilemma comes in: Should those who better understand Agile push it as a way of saving organizations, over and above letting leaders know it’s really not what they’re paying for? And how could the “Trojans” be so wrong?

Well, Scrum. (In a way Scrum was a Trojan Horse that killed Agile.)

Sutherland sold Scrum to managers as something that gets “twice the work done in half the time.” It’s an enchanting Trojan song. The Trojans, after all, think Agile is something that will help them meet their Waterfall numbers faster. This is how they’re paid to think — their pay is tied to output. More value might be generated by maximizing outcomes per least amount of work, but that doesn’t always get you a bigger bonus. (As the Yiddish proverb states, you can’t put a “thank you” in your pocket.)

The Scrum bill of goods resonates with orgs that confuse “productivity” with the busyness of individual contributors, indeed with orgs that still focus on “individual contributors” at all. The Greeks, of course, also want Agile, which they see as separate from the Waterscrumfall so beloved by the Trojans. They think they see inside the horse, and inside it is not only the death of Waterfall but of many of the structures that feed into it. So, you basically have two groups using the same terms while speaking completely different languages, often with radically different agendas, and we wonder why Agile transformations fail.

OK, so let’s talk about “power.” Agile “transformation” is an elevation of principles originally meant for dev teams to the level of org agility, meaning the very flexibility, adaptiveness, and intelligence of the system itself…and whether it will emerge fragile or antifragile. This opens a black hole swallowing what it formerly meant to do “strategy,” to “plan,” to “lead,” and little remains the same. Agile transformations often devolve into power struggles and, paraphrasing MacArthur, don’t so much die as just fade away…or fizzle out.

Does it have to be this way? In their excellent book, Inviting Leadership, Mezick and Sheffield offer a possible exit. Leveraging the emerging field of Authority Studies, they call for an updated understanding of the role of authority in organizations. Through this, one can come to see “Agile transformation” in a new light, shifting the focus to the value-adding distinction between “formal” and “informal” authority.

In the analogy above, both sides of the struggle are wrong. The Trojans are wrong in thinking the answer is to (only) increase Greek output. (Yes, output matters, but focusing only on output is tantamount to solipsism.) The Greeks are wrong when they argue the entire org should be Agile. (It turns out that’s not even possible.)

As Mezick and Sheffield put it, traditional leadership can no longer keep pace with the world. Indeed, it may be as much as 1000x too slow. Traditional leadership relies on “formal authority” — the structure captured in the org chart. As this implies, in today’s world the org chart never depicts how the work actually gets done.

Most real work gets done via “informal authority” — the self-managing allegiances and networks which emerge independently from the formal org chart. It’s common, Mezick and Sheffield observe, to speak of this pejoratively, wanting to challenge the “shadow hierarchy,” but this ignores the deeper reality that the informal network is a necessary workaround emerging to keep the system itself alive.

Informal authority handles complex problems with adaptive self-organization. As Mezick and Sheffield define it, self-organization is the dynamic (read “Agile”) sending and receiving of authority. When we find people we want to associate with, self-organization naturally follows, run by the currency of “street cred.” When we ask people in our group to make certain decisions, we grant informal authority on the fly. When we no longer respect someone’s decisions, said authority is revoked just as quickly. This is a real-time re-org. It is emergent, opt-in, always-on, and ever-informed by the collective knowledge of the group in an Agile way that formal authority can never match.

This doesn’t mean, however, that the Greeks are right who want formal authority replaced with something “Agile.” (Would the result be anarchy or holacracy? We needn’t go there. Yes, only a self-organizing team can be Agile, but all leadership cannot be self-organizing.) After all, the informal system has its problems too. Power lies in knowing how authority is distributed. In the informal system, this is both in constant flux and only tacitly known. There are no publicly-visible “rules of the game,” and that’s an issue. If that’s all there was, everything would be politics. The formal hierarchy is direly needed, but Agile does prescribe what its focus should be. To see why, let’s return to the “Trojan War.”

The Trojans typically just want the Greeks to increase throughput. Those who bother looking — such as Tim Ottinger — argue dev teams are seldom the constraint in the system anyway. As Drucker put it, “The bottleneck is at the top of the bottle.” The goal then shouldn’t be to make formal leadership “Agile,” but to remove this bottleneck. The bottleneck is caused by what we might call “decision authority hoarding.”

Agility, it turns out, has more to do with the distribution of decision authority and very little to do with the efficiency of dev teams.

Formal authority tends to want the entire system syphoning information up to decision makers. That is not Agile. Instead it should be delegating decision authority down to where the information naturally resides. Remember Andy Grove’s admonition: Decisions should be made by the lowest level responsibly possible. This is achieved by having the formal hierarchy focus on structuring the known “rules of the game.” They should focus on shaping the constraints the informal authority network then works within, granting authority to the informal network to make the decisions when needed (at the “last responsible moment”), so long as they are within the given constraints. (This also goes far in minimizing delay costs.)

Some at the top, of course, still won’t like this. (It means they need to stop authority hoarding.) That’s unfortunate, because we cannot keep pretending it’s the 1980s. The world has changed. As executive coach Sharone Bar-David puts it, “Power isn’t what it used to be.” The old view was that power came with certain “privileges.” Not so today. To see why, we must also distinguish between what Mezick and Sheffield call “legitimate” and “illegitimate” authority. In today’s world, even formal authority is servant in nature. Your day-to-day actions as a leader, your congruence or lack thereof, your presence, how you “show up,” continuously determine the very legitimacy of your authority, which must be continuously earned. Just because you’re in a position of formal authority does not mean anyone will take you seriously. (Just look at the President.)

“Leadership” is about making decisions that affect the group. Authority is only legitimate to the extent those affected by such decisions support the person making them. When the formal hierarchy ignores this key information flowing from the informal network, trouble will follow. At best, morale will be shot. At worst, there will be mutiny. The informal network will start to undermine the formal structure. Remember, this is of necessity — the informal networks are what get the real work done. When the formal authority hierarchy ignores this dynamic, it pours gasoline on the very subculture power struggle that tanks so many transformations.

To close, no organization can increase its ability to sense and respond in a complex environment by keeping decision authority confined to its formal hierarchy. Agile transformation, properly understood, is about keeping formal authority in place while empowering it to manage differently. As Rob England puts it, Agile transformation, at its very core, is about new ways of managing. It’s about new ways of leading. (Again, it’s not about dev team efficiency.) And how could it be otherwise? What is the point of management, after all, if not to help the org handle complexity?

So…how do we stop fighting the “Trojan War?” First, we stop trying to make the formal authority structure “Agile.” There is nothing about it that could ever be Agile in the first place. Second, the formal hierarchy, if it ever wants to move past lip service to Agile coupled with same-old, top-down Waterfall, must focus on enabling the informal authority network to make fast decisions, close to where the information lives, as it learns its way forward in a sense-and-respond (again, read “Agile”) fashion.