The Social Cost of Scaling Software — and Living With It.
“A system of cells interlinked within cells interlinked within cells interlinked within one stem, dreadfully distinct…”
—Baseline test, Blade Runner 2049 (2017)
What’s the problem?
If you work within a fairly large software development organization (say, >150), your organisational structure might have different supergroups, each composed of multiple teams. It might look like this:
Each supergroup would have a different mission and priorities, which branch off to multiple individual teams, each with a different mission and priority, composed of individual developers with their own internal rubric of missions and priorities —
You get the idea.
If you’re particularly lucky (or unlucky), the group might be developing to a single product — and in some cases, a single stack. This is hard, for reasons that are very fundamentally human. I’m going to spend some time explaining why.
A Thought Experiment.
Let’s say you (Developer Red) joined a team at the same time as another developer (Developer Blue), so you can work on the same things for say…5 years. You shared the same incidents, ran through the same sprints, did the same kickoffs, had the same leaders.
I would submit that at the end of that period, you and Developer Blue would develop similar priorities and sensibilities over time. Not completely of course, but there would be things that you both would hold as important.
This sounds glib, but let’s say we flip to the opposite. We made a copy of you — with everything that you are, everything you hate, and everything you hold important. But we split you off to two different teams for the same time period.
One version of you went through a team that had a lot of crunch, had leadership that indexed towards customer value as important. The other had a bit of flex and time to index for quality. Different projects, different leaders, different teams.
I would submit that at the end of that process, you would end up with two very different developers.
I guess what I’m trying to say is, our sensibilities and the things that we hold important are largely (and in some thought circles, completely) defined by our immediate environment, the constraints that we live in, and the people who are around it.
Why this makes things hard.
Unfortunately, a very fundamental aspect of software delivery is being in discussions with other people who have those different fundamental priorities and sensibilities.
Remember: each tribe would have their own priority, their own mission — so these conversations would be happening all the time.
The problem being human makes it vulnerable to human constraints. For some people, it takes a large amount of effort to empathise, reconcile, and understand another viewpoint. Whether it’s due to an avoidance of conflict, or the expense of emotional labor, people will end up creating groups who hold the same sensibilities. We like talking to people who aren’t a lot of work.
And if we’re not careful, we might accidentally end up with hard-drawn bifurcations of opinion (and perhaps even social groups). And if we’re really not careful, the social partitions that we have end up accidentally representing themselves in the software that we create. How many times have we seen different organisational groups with fundamentally different stacks or practices?
I use the word accidentally deliberately; there is a choice that software development organisations must consciously make when they get to a size that enables the above social problems. We see this all the time in different frames (Standards vs. Autonomy, Consistency vs. Enablement, Leverage vs. Independence) — which is essentially a question of: How much do we have to be aligned?
I don’t think there’s a “right” choice here — only a set of tradeoffs that would apply themselves differently in different organisational contexts. However, it is important that an active choice is made as it implies an understanding of the tradeoffs.
After all, if we don’t make a decision, then a decision will be made for us by circumstance.
A caveat.
Briefly: This is just one of the possible failure modes of large software organisations — one that I care about deeply. There’s a lot more, covered much more in depth, but we’re focusing on just this for now.
Social Gardening.
Regardless of the choice that you make, there will exist a layer to your software development process that needs that alignment. Maybe you want your engineering strategy to be flat. Maybe infrastructure platforms need to be consistent. Or maybe you made a choice to be aligned all the way.
This fundamentally human problem will exist proportional to the scope of how much you want to be aligned. And human problems need humanistic solutions. So how do you cope?
Cultivating Communication.
Communication channels specific to particular needs will need to exist, and will need to be sized well within the scope of alignment. But how do you guarantee that they work well?
My mental model for communication patterns is that you will get three shapes:
Contributive communications are where people have something to share, or add, but where the information does not need active assent or responses. These are the cases where people say that they’ve built a tool, or created a pattern:
While assent or responses are not required, you have a choice to celebrate the act of sharing. This reinforces to the community that the act of sharing is dope, and more shares will be similarly celebrated.
Interrogative comms are where people want a response to something, but no one in particular is required to answer. This is when people ask questions to the community in general.
Choosing to celebrate and highlight responses would a way to reinforce that the act of engaging is a net-good. This is a good way of cultivating communities, and encouraging people within different organizational units to do the same.
Decisive comms are where people want a response to something to which an answer is required. This is for your processes where a gatekeeper is required (Can I do X? Does Y make sense?).
While highlighting responses is good, you can also capture a lot of good by choosing to create clear, unambiguous processes. The act of collaborating is something that needs to be blameless, non-restrictive, and encouraged.
Fostering Community
If everything goes well, what you’re going to get is different “cells” with communication lines firmly established. A tech community is only as strong as the engagement of the sum of its parts: the people it is composed of. If you get it right, you’ll see it in the engagement whenever someone reaches out.
I personally measure it by the number of “Thank you for the help” messages that happen within a channel:
Safety becomes critical at this stage — you can choose to make sure that everyone has the right attitude when it comes to issues in a shared space. It looks like this:
One thing that I like to highlight is enabling safety within socio-technical circles also allows for safety of the output. This means that error states surface themselves well, and that no one feels afraid to say that things went wrong. This is a much preferable state than to find out issues tucked away, weeks after it already happened.
Default assumptions for others.
So now we our means of communication, and reduced friction to contribute to the community. But what about our baked-in priorities, the difference about the things we value?
This is one of my most overused slides across my career. I’ve found this frame useful when defusing frustration at things like bad implementations, rushed jobs, or an accumulation of tech debt. Everything is important, but some things are more important to other people, other projects, other circumstances. Everybody is doing the best that they can…
Default assumptions for you.
…even you.
The same rubric that we apply for other people also has to apply to ourselves. No one wants to turn up to work to do a bad job — we are all doing the best that we can with what we have.
Why does this matter?
An unfortunate fact of software development is that it is composed of a considerable bulk of things that we don’t talk in conferences or papers. What’s usually called “office bullshit” or “office politics” in aggregate is usually decomposable to a misalignment of priorities and tradeoffs and understanding, superimposed over delivery cycles that want the next cool thing as soon as available.
It seems so insignificant, when explained like that.
But there’s a real, incremental, drudging effect when those misalignments happen meeting after meeting, day after day after week after month after year. There’s a real weight to the soul, to which the only antidote are these active decisions to engage, encourage, and empathise.
Now, I don’t mean to come off as someone moralising — because most days, I forget to do these things, make these active choices. Some days, I actually don’t want to.
But if you happen to find yourself in a space where you have the capacity to make that decision, it’s the difference between someone who is tired and angry and cynical and wary of the rest of the world — or someone who accepts the world for what it is, and chooses instead to cultivate, to grow, to hope and work for the better.
“…and against the dark, a tall white fountain played.”
— Baseline test, Blade Runner 2049 (2017)
