“Shame Driven Development”— When and how Bot-ing is better than Blocking
A little bot called Snitch taught us that visibility is more effective than forced barriers. Here’s how it leads to better enforcement and why, in the right context, shaming can be productive and hilarious
We all know shaming is a bad practice that promotes bullying. But shame itself is the outcome of one’s frustration of not complying with (own’s) code of conduct, where shaming can also be thought of the act of surfacing that understanding.
The nature of rules and shame
Rules draw the boundaries of our code of conduct, and in development, as in real life, we often use some sort of forcing mechanism to enforce them. For example, you shouldn’t merge a branch if it doesn’t have passing tests. In our digital environment, enforcement is easy — simply block the branch in GitHub or revoke permissions from anyone except for the chosen one(s).
We tend to add tons of restrictions as this is the safest, most intuitive way, but it comes at a price. Digital barriers are like gates and walls in the real world — they’re efficient, but they may quickly work against you.
In the real world, you sometimes have to bend the rules (you would jump over a locked gate to save the owner from fire). Other times the rules are redundant for specific tasks (what if the owner herself forgot the keys?). Same thing with digital restrictions.
If you add an exception or allow certified persons to override the restrictions, they quickly loose their hermetic effectiveness. Rules must be enforced in a strict way and applied equally to all cases and persons in order for them to be effective. So maybe there's another way?
Don’t block, just snitch — a use case
As part of our code of conduct, all pull requests (PR) must pass a code review (CR) before being merged to master. That’s a basic practice in most teams. When that rule was first introduced, many developers ignored it from time to time. That led us to force a status check in our main Github repo. That was the first wall we’ve built.
But Github allows administrators to overcome this restriction, and since most of our developers have admin rights (or let’s say — our “troubled” developers have), that didn’t change anything.
So we added another “wall” and checked the little box that says “Enforce all configured restrictions for administrators”. That worked great. Except for when it didn’t, because admins could simply go into settings, uncheck that box and then check it back on when they’re done.
Still, It was better because this process takes a few clicks, and developers used it less since they were lazier than impatient. I thought it was the best solution because if we can’t block the process all in all, at least make it hard so they only do it when they have to.
And that’s the thing — sometimes they have to.
For example, a DevOps guy working at night updating configs might need to push multiple commits to get things up and running. Same for when updating documentation files. There are many cases in which requiring a CR (code review) just makes things complicated.
Another question arose and it is — how do you deal with a rogue developer? Of course, you can frown and scream, but you won’t really fire a guy for skipping code review. Besides, what kind of a culture does it promotes if not one of hiding and excuses.
Turning it around
First I thought of writing a bot that will notify me of any developer that removes the restriction, so I could at least keep track and know if the removal was inevitable. That’s another wall (3rd one!) to protect the same rule. Unfortunately, or luckily, that’s not possible with current Github API.
So I decided to tackle the issue from a completely different angle.
We all know that sometimes you can solve problems by going along with the power that drives them, instead of conflicting it (think of Judo vs Karate).
Sidenote: A textbook example is that of cooking gas. Liquefied petroleum is odorless. In order to fight gas leaks they’ve used tons of security measures, but it was never enough, as gas always found a way to leak. So a very smart guy decided to mix it with Ethyl Mercapta, another gas, one that gives cooking gas that familiar stink. That sounds obvious to us now, but it is actually quite counter-intuitive — to fight the dangers of gas…you add more gas to it.
So I turned it around and disabled all locking mechanisms regarding CR and allowed developers to merge PRs to master without any review. But I also wrote a quick little Github bot that tracks merges and checks to see if they were approved by someone who is not the creator of the PR.
When it detects such an event, it sends a message to our main Slack channel. We called it SNITCH bot and it revealed a very interesting thing.
Shaming can be fun
When developers knew their actions were being published, they were less keen to skip the rules. It’s not that anyone actually felt a shame, but visibility changed something.
Not only that but, since snitching was equally enforced on all, executives, team leaders, and juniors alike, it promoted greater personal responsibility, eliminating the question of who guard the guards.
Snitching made the developers think in a different way — I can say from my own perspective as a developer, that it shifted my thoughts from trying to overcome limitations to thinking about how may I publicly justify myself.
Instead of focusing on the rules, I am focused on the outcomes of my actions.
As an extra bonus, the event is often followed by humor from members, which promotes a much more positive environment than the frowning face of a team leader, asking yet again the rhetorical question “but why’d you do that?”.
Taking it an extra step
This use case grown even further as my point of view shifted and I started questioning how ineffective are other blocking mechanisms, some that I took for granted.
A much more bold example is forcing all tests in the CI to complete on a branch before merging. This is a core rule everywhere, as you never break master. But if you think about it, what’s the price you pay for that rule, and what harm will it do if you remove it?
Basic CI flow takes time and for large systems, it takes a lot of time. This time costs money, and I mean it literally — CI servers are expensive. But the real problem is deadlocks. When a branch is merged, it forces other branches to be updated, leading to another CI run, and another. And since we are working intensively, merging multiple times a day, this may lead to deadlocks where PRs are stuck in the queue, endlessly updating. But we must run tests! Aren’t we?
Let’s say a developer already saw her tests green once, and now updates the branch by pulling master in. If she merges it, the main branch may break, but chances are that it won’t. And if it will, you can simply revert it. This is another wall we can break down and replace with a snitch. Talking about shame —it is safe to agree that no one wants to be the one who broke master.
Removing the block, not only reduces friction, it actually makes the developers more aware of their actions, and saves expensive CI time and money.
When should you block?
In the end, I came to a conclusion that the only rules that should be strictly enforced by blocking are rules that saves the developers from themselves. Preventing them from doing a harmful action that is the result of unseeing important info. For example — we enforce a PR title to comply with conventional commit format. Or prevent a merge of a branch with WIP label. Or even preventing the commit of unwanted changes in files that slipped in.
These blocking rules enhance developer’s productivity and the proof is that no one ever tries to unblock them.
The result of those decisions proved correct — we can clearly see how there are significantly fewer (if any) illicit actions being made after the block was replaced with a snitch than it was when they were actively enforced.
Take a minute to think, what other rules and regulations can better be enforced by visibility instead of blocking?
