Intro to Mastering Chaos — Overcoming Conway’s Law

Josh Evans
Aug 28, 2017 · 6 min read

As engineering leaders we constantly confront apparent systemic chaos. Whether we’re designing a scalable microservice architecture, working within a large organization, or vying for position in a significant, competitive market it’s impossible to fully know “the system” in all of its complex glory. What are the elements that matter? How do they connect to, support, or obstruct each other? How do they fail and what do you do in response? And the most interesting question of all — how do we influence such a system to achieve a desired result without full knowledge or control? In short, how do we master chaos?

In the face of incomplete knowledge and control some might be tempted to throw up their hands in despair. But successful mastery of complex systems requires tenacity and the ability to see the forest ‘for the trees. Thus, when solving for specific challenges in specific systems, it helps to have a method or angle of attack to break down the problem space and surface solutions.

For the sake of this discussion we’ll borrow a page from WWII US Air Force Colonel and military strategist John Boyd. The OODA loop (observe, orient, decide, act) is a decision cycle, developed by Boyd that proved effective in training and evaluating fighter pilots. Those who executed the loop quickly and effectively had a much higher survival rate. In our case we’ll repurpose the OODA loop to solve for systemic problems, hopefully with the same success. To make this concrete let’s work through an example - Conway’s Law, a common, systemic challenge for technical organizations.

Conway’s Law

In 1967 Melvin Conway coined this insightful adage:

organizations which design systems … are constrained to produce designs which are copies of the communication structures of these organizations.

or to put a fine point on it:

If you have four teams working on a compiler you’ll end up with a four pass compiler

Conway’s Law describes how organizational concerns bias and constrain architects and resulting solutions.

Conway’s law is pervasive. It describes a systemic dysfunction that exists in virtually every company. It doesn’t take a degree in psychology to see that Conway’s Law is rooted in human nature. More specifically ambition and tribalism. In other words, the challenges presented by Conway’s Law arise from a focus on self or loyalty to team at the expense of the larger community or company. At best the solutions that result are suboptimal. At worst they are deeply flawed and dysfunctional, working against company interests. And it’s easy to see why.

If the first question when solving for critical business challenges isn’t “what’s in the best interest of my company?” (before my team and before me) then excellence will be hard to achieve. Politics and personal agendas rule or at least throw grit in the gears. If key players are taking a territorial stance then it doesn’t matter how awesome your new architecture is or what problems it will solve. You’ll run into resistance, potentially derailing your best laid plans or imposing unwieldy compromises on your design.

Now that we can articulate the symptoms and have a theory around root cause let’s do some thoughtful analysis.

Hamilton Helmer in his book 7 Powers: The Foundations of Business Strategy says:

Part of the tradecraft of economists consists of teasing out the essential nature of a problem by more formally modeling it. The art lies in selecting the simplifying assumptions: they must be chosen in a way that isolates the salient features of a problem, while not assuming away core characteristics.

Given that economies are (extremely) complex systems this statement is quite relevant to our discussion. So — what are the salient features of Conway’s Law? What are the simplifying assumptions? How do we model the problem?

If we take a big step back Conway’s law simply describes the relationship between two complex systems— computer-based software architectures and human organizations. At this altitude specific architectures, organizational structures, and participants may be completely irrelevant. These “trees” distract and prevent us from understanding root cause.

The “essential nature” of Conway’s Law is power and control, where one complex system dominates and constrains the evolution of another. It is triggered when challenges and resulting solutions span organizational boundaries. More specifically, it is triggered when solutions may have a significant impact on organizational structure, roles, and responsibilities. This is where ambition and tribalism come to the fore.

Now that we can articulate a trigger, a cause, and an outcome we can form strategies and plans.

We can overcome Conway’s Law with a (deceptively) simple two-part strategy:

Invert the Power Dynamic

If the fundamental dysfunction associated with Conway’s Law is that organizational structure holds sway over architectural designs then the solution is to invert the power dynamic. Put architecture first. In other words, during critical phases of analysis and design we must treat organizational concerns as irrelevant.

We must create a virtual “clean room” for engineers to freely explore technically viable options and design the best solution based purely on business and technical requirements without organizational constraints. Once the solution has been technically vetted and refined organizational perspectives may surface practical considerations. However, proposed deviations from optimal technical designs should be viewed skeptically through the lens of Conway’s Law.

Vet for Selfless Leadership

Inverting the power dynamic is a useful tool for those who want to understand and overcome the implicit biases that organizational thinking naturally imposes on us all. In other words, success depends on the intentions of those participating. It assumes that leadership is selflessly motivated to do what’s best for the company to achieve strategic goals. But very few (if any) of us are 100% selfless and detecting selfish or tribal motivations is not always easy.

People will rarely admit (or potentially even realize) that they are being selfish or territorial. Much of the time the bias is implicit. As a result, successfully inverting Conway’s Law requires explicit cultural alignment and incentives that drive desired behaviors. Cultural values should focus on explicit valuation of selfless behavior, evaluation of motivations, questioning territorial positions, and rewarding those who solve problems across organizational lines. Incentives are ultimately about who you hire, fire, and promote.

As mentioned earlier, the methods described above are deceptively simple. Implementation can be difficult. If your company or organization doesn’t explicitly value solutions over organizational politics then there may be a significant uphill battle. But the only way to effect change is to try. Here is one possible approach in three parts.

Raise Awareness

Discuss Conway’s Law in the abstract with other leaders. Frame the problem as a simple power dynamic between organization and architecture divorced from specific challenges and implementations. Very few people believe (in the abstract) that organizational structure should limit solutions to significant challenges. It’s only when a specific situations arises that things get personal or tribal. So start with philosophical alignment.

Provide Examples

Find concrete examples where organizational concerns led to suboptimal solutions at your company. Estimate the increased complexity or diminished value presented by actual vs. optimal solutions to underscore the cost of organization-first solutions. Every tech company has examples to draw on and estimated impacts help others internalize the opportunity presented by an architecture-first approach.

Seek Partnerships

You don’t have to change your entire company to overcome Conway’s Law. It only requires participation from the organizations affected by a given architectural change. Seek out opportunities to partner with a like-minded peer. Kick off a cross-functional project by explicitly discussing Conway’s Law and how it negatively affects solutions. Give your architects and engineers explicit freedom to draw outside the organizational lines and see where it leads.

And, of course, share your stories so that others can learn from your experience and example.

Note: see Refactoring Organizations: A Netflix Study to learn more about overcoming Conway’s Law.

Where do we go from here?

The purpose of this exploration of Conway’s Law is to provide a simple example of rational, systemic thinking, the formation of strategies and methods to systemic problems and resulting solutions.

The right strategies can make or break an organization or company. They inform what you do and, more importantly, what you don’t do. Bold investments present the risk of bold failure. Inaction can be equally dangerous. We must have the right tools to identify and place good bets. Stories in this publication will focus on growing that toolset with an eye to improving problem solving and decision-making capabilities.

Mastering chaos is a journey and we’re just getting started.

Mastering Chaos

Welcome to Mastering Chaos! Our charter — to share effective strategies & methods for managing complex systems. Here we’ll delve into a myriad of technical and social systems, addressing challenges that leaders face every day.

)

Josh Evans

Written by

Mastering Chaos

Welcome to Mastering Chaos! Our charter — to share effective strategies & methods for managing complex systems. Here we’ll delve into a myriad of technical and social systems, addressing challenges that leaders face every day.

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade