Conway’s Law: How to dissolve communication barriers in your API development organization
One mistake that we often make as architects is failing to consider how the patterns of communication within our teams and organization as a whole affect the pattern of design on which we construct our product.
Melvin Conway said it better in his April 1968 paper “How do Committees Invent?”:
“… organizations which design systems … are constrained to produce designs which are copies of the communication structures of these organizations.”
This statement has come to be known as “Conway’s Law.” Eric S Raymond gave us a little more specificity in The New Hacker’s Dictionary, paraphrasing the initial paper:
“If you have four groups working on a compiler, you’ll get a 4-pass compiler.”
In March 2008 Alan D. MacCormack, John Rusnak and Carliss Y. Baldwin of Harvard Business School published “Exploring the Duality between Product and Organizational Architectures: A Test of the Mirroring Hypothesis.” (The Mirroring Hypothesis is a restatement of Conway’s Law).Their findings reinforce Conway’s initial thought in that:
- Organizational distribution and product modularity are correlated
- Products really do end up looking like the teams that create them.
How does this affect our development process as individuals and teams, though?
Microcosm to Macrocosm
If you’ve ever programmed in a less-than-perfect collaborative environment, you’ve probably noticed that your code ends up more closely meshed with the teammates you talk to the most. Let’s look at how you collaborate with Stella (Your Bud). Your desks are side by side. You exchange thoughts as you code and help debug each other's work. Not only do your parts of the project work well together, both of you know how your dependencies are built.
This stands in sharp contrast to your interaction with the “rockstar” developer that submits for code review without ever having talked to their team. Although their code works, it links to yours only by way of the agreed technical specification and is otherwise a black box.
Before you judge the rockstar, consider how many changes You and Stella made that went undocumented because talking was faster.
If everyone works like this, product architecture begins to follow seating charts rather than UML. Individual actors assume that their small undocumented change is the exception in a sea of impeccable records. In reality, they are creating technical debt, adding complexity and leading to bit rot. When scaled between them or across whole organizations, the entire architecture of a service or product is affected. Undocumented changes and assumptions compound up through an organization, until not just individuals but teams themselves are making decisions that no one else has visibility to. When higher level strategic decisions are made, they rely on inaccurate commonly available information. Be it a lone programmer, a team, or a whole department — all parties both contribute to and suffer from misinformation. Correct organizational structure establishes clear lines of communication.
Departmentation
Departmentation is the foundation of organizational structure.Whether you are a CTO, manager, or team member understanding the reasoning behind your organization’s structure is important. What structure lets you deliver your best work?
Departmentation: the process of departmentalizing an enterprise for gaining efficiency and coordination : the grouping of tasks into departments and subdepartments and delegating of authority for accomplishment of the tasks
Some of the largest problems affecting software development revolve around departmentation:
- The right work being done by the wrong person (E.G. your front end engineer maintaining AWS)
- The right person in the wrong role (E.G. A brilliant DevOps engineer running your automation team.)
- The right functionality being maintained by the wrong team (E.G. QA Documenting your API)
- The right information known by the wrong people. (E.G. The cardinal sin of having only a single person that understands your whole system)
Often these issues arise from time constraints around release, which is a planning and scoping problem. As strong as the urge is to keep doing something yourself because you don’t want to surrender control, you probably have more important work to do that is closer to your role.
Lets look at how three classic departmentation strategies work:
- Functional Departmentation
A functional organization attempts to tackle the issue of communication by grouping teams according to the business or technical functions which they support. Each function delivers independently of the others but heavily relies on visibility into other units to support their work.
2. Product Departmentation
A product-oriented organization is suitable for companies producing multiple products or operating on multiple platforms. Functional grouping of jobs and resources form around the products or product lines. Often times products have a common infrastructure that may need to be maintained by its own team.
3. Process Departmentation
A process-oriented organization passes products through different production stages. Each of these stages is a department. Departments are headed by experts in the specific stage. Each business unit informs the unit before and after it. All working together create the end product. These units remain static across multiple product creations and releases.
The Inverse Conway Maneuver
“The ‘Inverse Conway Maneuver’ recommends evolving your team and organizational structure to promote your desired architecture. Ideally your technology architecture will display isomorphism with your business architecture.” — ThoughtWorks 2015
The goal of the Inverse Conway Maneuver is to move both your technical and organizational architectures towards the same isomorphic shape.
Isomorphic: corresponding or similar in form and relations.
In order to leverage Conway’s Law to our advantage, we have to start from both the technological and the organizational ends and work inwards. We’ll need to consider tools, and then combine them with the communication structures we’ve already discussed.
Tools Mirror Your Processes
There are a plurality of tools to help managed communication around technical product development. I said earlier that tools can’t fix people problems and I hold fast to that. But a hammer is faster to drive a nail than a shoe. If the shoe fits, the right tool can take facilitate a change in people process. Technology abbreviates communication.
If you’re not familiar with workspaces, it’s Postman’s collaboration feature, which sorts work into shared (or personal) common space, like a group folder in Google Drive. workspaces live under Teams, which include the whole of an organization utilizing Postman. Workspaces are visible to every member of their team unless made private (an enterprise feature). So even if I develop APIs to interact with my company’s backend, I can still pop into the reliability team’s workspace to see if I can leverage anything they are working on.
At Postman we’ve seen teams visualize and change their structure via workspaces. As with most collaboration tools, workspaces removes the barriers of access to files — in our case collections. collections are an executable format, they can be understood with minimal communication between creator and consumer. As a result you can experiment with a dependency in the same way its creators would at every stage of the build process.
Other Postman Features help turn collections and workspaces into full fledged information warehouses that can preempt the need to walk over to someone’s desk or send them a slack to ask for a test run or info on how to use one of their APIs. Teams may create collections in other workspaces to enforce contract tests so that changing dependencies don’t cripple other services. Monitoring facilitates these tests to run regularly to ensure everything is still copacetic . If something isn’t, integrations for services like Slack and PagerDuty exist out of the box to alert you that something’s broken. Where collections let you learn about another team’s service by executing and playing with it, documentation is auto-generated so you can read up to clarify things that may not be apparent through fiddling. If you want to make sure you aren’t reinventing the wheel, you can use API search to find functionality someone else may have already built.
Workspaces Mirror Your Org Structure
How an organization structures its workspaces determines their mileage. Sound familiar? Let’s look at some strategies for workspace creation. First, some of the organizational structures from earlier reproduced in workspaces:
In function workspaces, teams like DevOps and Backend are tightly coupled, and will frequently test and share between each other’s workspaces. Support needs to be able to dip into each of the other three business units’ for documentation. Front end needs access into Backend’s workspaces to build on their APIs and establish contract tests.
Each product operates as a separate functional business unit, APIs and documentation for platform interaction (iOS, Android, Windows, Web) are shared knowledge. Although these teams are independent and self contained, talent may often move between them, and they likely share common architectural resources through collections. Integrations between products rely heavily on automated tests and contract monitoring through collections in each other’s workspaces. It’s common to use the API search feature to for discovering helpful or insecure APIs from other teams, which also cuts down on duplicated work. Tagging different teams within the individual products (Front End#Facestagram) can reduce confusion and clutter, as we’ll see later.
Collections are shared between process workspaces to represent “definition of done”. Build knows it has fulfilled its requirements when it can run the DoD collection from Design without any test failures. QA /Test will share bug collections with the Build team. Release can then use collections from the other teams’ workspaces to verify a new update before it’s pushed.
The solution of just recreating our org structures from before in workspaces may be simple, but it’s an efficient way of knowing what’s going on in each part of your organization simultaneously.
Here at Postman, we go a few steps further, tagging workspaces and collections for easier filtering. With collections, there is a base idea of “functional” and “descriptive” collection formats. One serves as documentation, and the other actually takes actions — i.e. is designed to be run with collection runner. We do this with workspaces as well.
When we combine workspaces that channel and improve our organizational structures with tools that circumvent the frustration or miscommunication of our processes, we are making Conway work for us. By focusing on moving both architectures together via the Inverse Conway Maneuver we can reach the ideal isomorphic state:
- Information flows naturally between related teams
- Process and architecture are informed by team structure
- Teams and managers are suited to one another
- Team has a holistic view of organization
- Informed and capable support organization
- Less bugs on release
- Faster time to release
- Work is rarely reproduced
While Postman is an excellent selection for resolving some of Conway’s woes, there are a gajillion other tools out there to help streamline your communication. I’d love to hear about how you’ve solved your own problems in the comments below, and don’t forget to subscribe to Postman Engineering!
