Living in the Age of Software Fuckery
Ten Anti-patterns and Malpractices in Modern Software Development
1) Business Thinking Can Favor the Short-Term — To an Extreme
According to Alan Kay, there are legislative problems in the US that effectively force businessmen to make myopic decisions — https://www.youtube.com/watch?v=NdSD07U5uBs#t=36m12s
Some propose that the phenomenon of easy credit and low-interest rates can make it economically nonviable for a company to act in its longer-term interests. Some people just blame capitalism for its inherently exploitative nature.
Whose opinion is most accurate is probably immaterial for this discussion. What matters is that your manager, and all the way up to the CEO, can spend very little time thinking about, much less preparing for, distant future engineering concerns. This is for the same reason that a fire fighter blasting a water cannon doesn’t think how much water damage he may be causing — the goddamn building is on fire.
Anyone whose eyes are not glued to an IDE for 50+ hours a week is effectively blind to the company’s metastasizing engineering problems.
But software works in the opposite way. The longer-term you can think about your software, the more robust and less costly and painful its development will be. The more time you put up front into building simplicity into the system with intelligent layering, the less of a nightmare it will be to develop.
Because of the short-sighted way in which management usually thinks (again, likely due to the incentives provided by the larger economic environment), the integrity of your software system isn’t usually on their radar. If they notice it at all, they’ll more often see it as an obstacle to work around rather than a constraint to respect.
2) Being Crucified for Not Being a ‘Team Player’
It won’t always matter how productive you are, how many late nights you invest, or even how well you actually collaborate with your fellow engineers. The moment you engage in thinking misaligned with management’s above view, you may be put up for crucifixion on this cross. And if you’re the type of person who relents to plain reason, you will eventually find yourself voicing legitimate engineering concerns.
You will feel the room temperature rise every time you do so, for you are stepping into a fire.
Ultimately, your job is to help management meet expectations for the next quarter. This means that most of your time may be spent fighting fires lit from the previous quarters, hunting bugs that could have been easily prevented, or refactoring several unrelated pieces of code just enough implement the next broken feature.
3) The Software Architect is Often a Politician
Ever wonder why you rarely, if ever, find yourself in a maintainable commercial code base? Surprisingly, it’s not at all due to a lack talent in the software world. There are tons of great software architects just biting at the chance to build the next great system.
The problem is that the main qualification for being a software architect is not usually technical wherewithal. The qualification for being a software architect is often in having the ability to force front line engineers to implement functionality at an unsustainable pace. For this is what is required to satisfy management that is too often short-sighted.
When it comes to building a coherent, understandable, and well-designed system, the software architect may not have this as his top priorities. This may be true regardless of the long-term necessary benefits of doing so.
The architects primary jobs are usually to a) appease the business people to whom he is beholden, and b) engage in any manner of political maneuvers needed to force the engineering team to follow suit.
If our architect were to grow sufficient backbone to stop producing at the rapid-fire, unsustainable pace as demanded by short-term interests, he’ll eventually often be replaced by someone more malleable.
4) Suffering is Often the Commodity That you Provide
You might be under the impression that what makes you qualified for various positions in software development is primarily your technical acumen and ability to work with other technically-capable engineers.
In too many modern environments, you‘d be wrong.
While a certain minimum of capability is required to do your day-to-day work, what your value usually consists of is in grinding yourself against the piercing pincers of elusive bugs and razor-wire bundles of bullshit code until something resembling progress is made. You are rarely a problem-solver, but rather a problem-endurer.
Oh sure, you can solve the endless little meaningless problems that plague the day-to-day life of programmers everywhere, but the source of these problems — defects at the engineering level — are usually untouchable. Also untouchable are the systemic issues that cause each of these should-be-trivial-to-fix problems into 5–8 hour tasks. The interesting problems that have long-term engineering and business impact, the problems you went to college to solve, might not even reach your radar.
Rich Hickey gave a powerful talk on how to reliably engineer most of these problems out of existence — http://www.infoq.com/presentations/Simple-Made-Easy. Watch it in full; you’ll be inspired. But despair that, in most contexts, these are the engineering techniques you won’t get to apply. The business team often can’t be made to see the value of sacrificing a little time and velocity up front to improve engineering approaches.
5) Your Peers Often Won’t Support Change
Businesses and management can be short-sighted. But sometimes, their shortsightedness can be dwarfed by your coworkers. Whereas your manager sometimes can’t think beyond the next reporting period, your co-workers sometimes can’t think beyond the next pay period!
Because human beings are a coping species, within them lies a mechanism for making whatever justifications are necessary to reach their primary goal of immediate survival. They are genetically programmed to contrive whatever justification best support their immediate needs.
There is a whole class of literature of self-deception and what motivates it, but I’ll just leave couple choice links here -
6) Someone is Usually Ready to Undercut You
And the guy who is going to replace our newly-spined software architect? He’s the guy sitting across from you who never complains about the code, never wastes his time cleaning things up, and is always the first to shut down ‘toxic’ discussion. He’s the one whose ego has him up to his eye-balls in debt on consumer goods. As to thinking in terms of the next pay-period, he’s already adjusted his spending habits for his next promotion!
If you think your team’s code base is fucked now… just you wait till this guy is put in charge.
7) The Conscientious are Often Disenfranchised
It stands to reason that an engineer with sufficient seniority could put a stop to this circus. After all, code is as much about experience as it is about expertise.
Perhaps this used to be the case, but now becomes less likely. Today, we have something called ‘collective code ownership’. The reason given for this practice is at once a truth and a lie. Collective code ownership is to keep people from being ‘blocked’. After all, as we are constantly ‘coached’ by people with three weeks of management training, everyone is allowed to change any code, in any way, so long as it helps them meet their short-term goals.
If our most dilapidated code bases can do anything well, they can teach us viscerally the meaning of the term ‘the tragedy of the commons’ — https://en.wikipedia.org/wiki/Tragedy_of_the_commons
Putting even that valuable lesson aside, let us consider that being ‘blocked from making changes’ can actually be a very good and necessary thing. It’s something that the senior engineers of yesteryear once had the power to do. They sometimes did it out of spite, yes, but more often to keep code over which they had stewardship from being compromised by short-term thinking. Blockers were put up to ensure that software could be developed at a sustainable pace with the minimum of human suffering involved, and to be used as a check and balance against a management team not in a position to understand or learn about engineering trade-offs at stake.
In most cases, no longer.
8) Methodologies Are Management Tools
Which brings me to my next point. They’re not necessarily there to help you create better software, and if they are, they are often be warped into the opposite.
Agile has been the perfect example of this. What started out as a reasonable means to develop software has been turned into, as Mike Judge calls ‘Psych 101 MBA Bullshit’ -
9) You Will Think your New Team / Project is the Exception
Sometimes naivety is good, especially if you’re behind on bills from being out of work. In fact, lying to yourself can have serious short-term benefits. Such habits are probably so ingrained in you that you often decide to do so unconsciously. The term for this is being on one’s ‘Honeymoon’.
But if you’re going to lie to yourself, at least admit that’s what you’re doing! The more deeply you believe a lie, the more rude your eventual awakening will be. And then you’ll find yourself spending hours writing articles like this and raging against the whole blind stupid world on Twitter.
10) It’s a Race to the Bottom — and Few Actually Give a Shit
The truth is, commercial software development is often a race to the bottom. Whoever can trade away the optimal amount of personal integrity and endure the most suffering usually wins.
Management may think that you’re just paying your dues to be a part of their club (even if you have no interest in management), so they don’t give a shit.
Your peers often adopt a macho attitude against it all, and can come to see your frustration as a liability.
The average Joe thinks your life is puffery compared to his, so he doesn’t give a shit. He gives even less of a shit because you and your hipster brethren are jacking up his landlord’s property values.
In fact, you’re lucky to find a person who can admit to the above problems in the first place. Most people just convince themselves that nothing is wrong, so that they don’t have to carry the burden of caring. Few dare broach subject in public, much less at work.
What is to be Done?
Pull requests welcome.
But in all seriousness, we can start by telling the truth about the situation, to others and especially ourselves. At the very least, it should help to lessen the shock of the reality to people entering our field. The people most affected by this shock are the honest young people who are least suspicious of this type of chicanery. In other words, the people we lose the earliest are the people we may need the most.
Limit your exposure to the radioactive parts of the software industry.
This can be done by preferring short-term contract work.
You may think that this is a problem due to introducing gaps in your resume. Maybe this was a problem when your father was slinging code, but it’s not going to be for you. Again, companies are thinking short-term. Due to their shortsightedness, many cycle through employees as fast as your local burger joint. The last thing they care about is whether you will be at their company 5 years from now. Their only question is often — can this guy help us meet our next quarters’ expectations?
After many months and 35,000 views (EDIT — now 85k!) of this article, I’ve decided to forward-link my 3-part article series detailing proposed solutions to the above issues.
While some of the ideas I propose are new and untested, I think they’re worth exploring for people who are interested in these issues.
An Impromptu Interview about Sovereign Software Developmentmedium.com
While these ideas may be imperfect and non-trivial to execute in practice, I’m hoping that this type of ‘post-agile’ thinking will kick into gear the changes we all know we need!