On the accumulation of technical debt
Working in a team that consistently ignores and accumulates tech-debt is like living in a share-house where no-one ever cleans up.
What is tech-debt?
“It is not the business of the botanist to eradicate the weeds. Enough for him if he can tell us just how fast they grow.”
— C. Northcote Parkinson in Parkinson’s Law
When developing software, especially in a commercial setting, programmers, and more often those who seek to manage programmers, will insist on cutting corners rather than doing a job properly. The time-savings get chalked up to what has become known as “technical debt”. The term itself was first coined by Ward Cunningham, inventor of the wiki, as an analogy to fiscal debt; the short-term benefits are like a loan that must be repaid with interest in the form of additional work developers need to do later to work around the issues created. Because brevity matters, the term “technical debt” quickly became ‘tech-debt.’
Tech-debt is what economists refer to as an externality; a cost that is not factored into the price of a good or service.
Why do projects accrue tech-debt?
“Technical debt usually comes from short-term optimisations of time without regard to the long-term effects of the change.”
— Aaron Erickson in Don’t “Enron” Your Software Project
Not all debt is bad as anyone who has ever bought a house would know. Borrowed money can be used to purchase a high-cost capital item that will quickly begin to generate revenues. So too with tech-debt.
It’s a truism in software development, especially for startups, that ‘shipped is better than perfect.’¹
There will always be times when it makes commercial sense to cut some corners, whether that means skipping some unit tests, or opting for a quick brute-force solution you know has scalability issues that will bite you when your project has real users. Part of the allure of accruing tech-debt, especially for non-programmers making the decision to mortgage their business’ future, is that those accruing the debt are rarely the ones who have to repay it.
Individuals choose to accrue tech-debt but it’s the business that pays the interest.
How much tech-debt is too much?
“Most people know they have technical debt in their code bases, but if you ask them how much, they struggle to quantify it. If you ask many developers where the technical debt exists, they’ll point to the part of the code base they dislike working with the most; but if you ask them what the financial impact is, while they may wax lyrical about the evils of technical debt and how it slows development, they don’t have a quantifiable answer.”
— Glenn Bowering in Mapping and Costing Technical Debt
Parkinson’s Law of Triviality humourously posits that members of an organisation give disproportionate weight to trivial issues. This ‘law’ is often cited by those keen on accruing tech-debt who tend to deprioritise cleaning up of code. Notions that ‘tests just slow down development’, or that the NP issues within a brute-force algorithm are a ‘nice problem to have’ because they only appear when the business has ‘too many’ customers, can seem attractive; especially when the person in favour of the short-cut is not the one who will have to repay the debt.
“For a manager, a code base high in technical debt means that feature delivery slows to a crawl, which creates a lot of frustration and awkward moments in conversation about business capability. For a developer, this frustration is even more acute. Nobody likes working with a significant handicap and being unproductive day after day, and that is exactly what this sort of codebase means for developers.”
— Erik Dietrich in The Human Cost of Tech Debt
When your project has accrued so much tech-debt that it’s embarrassing, your increasingly unhappy developers will start to leave. Your most experienced developers will be the first ones out the door as they seek new opportunities and projects that are unencumbered. This kicks off a death-spiral for any project. Anecdotally I have observed a correlation between programmer absenteeism and high levels of tech-debt.
High levels of tech-debt comprise a barrier to entry for new hires who have to learn how to code around poor legacy decisions. This makes new programmers unhappy.
Given the importance of software to the working of the modern world, it’s arguable that paying down tech-debt imposes a massive burden on the economy as a whole.
How much interest does the business really pay on the tech-debt you choose to accrue?
It’s been alleged that up to 80%² of software development budgets are spent on software maintenance. The willingness to accrue tech-debt is the single largest contributor to this.
As Martin Fowler explains, “We can choose to continue paying the interest, or we can pay down the principal by refactoring the quick and dirty design into the better design. Although it costs to pay down the principal, we gain by reduced interest payments in the future.”
It’s impossible to estimate the interest rate that will be applied to the tech-debt you accrue.
“If the short-term optimisations that occur most of the time had a minor effect on maintenance (something like a 6% mortgage, say), such decisions to take on technical debt would be just fine. However, we frequently allow software development organizations to take out technical loans that, if they were transparent, would throw the corporate treasury department into a spiral of panic.”
— Aaron Erickson in Don’t “Enron” Your Software Project
Incurring debt when you don’t know what the repayments will be is akin to being a gambler borrowing from a loan-shark. Would you willingly take on fiscal debt at an unknowable interest rate?
Stephen Freeman argues persuasively in Bad code isn’t Technical Debt, it’s an unhedged Call Option that even if it is more expensive to do things properly, doing so reduces risk. As any business person ought to know, risk is always measured in dollars.
All debt carries risks but you can mitigate them by keeping your tech-debt levels low so you have breathing room to take on debt when it’s really useful to do so. By keeping your debt periods short you reduce the risk that the interest will blow out.
For green-fields projects there are plenty of ways to minimise the need to accrue tech-debt. Many of these can also be retroactively introduced into a debt-laden project as well but accruing tech-debt quickly becomes an issue of developer culture.
Like gambling addicts who will keep on borrowing despite being in over their heads, teams used to accruing tech-debt will just keep on skipping writing tests, avoid maintaining documentation, and dancing around the need to refactor. And also like problem gamblers they will become expert in rationalising their decisions, and explaining away problems as being trivial, or non-issues.
Once bad behaviour becomes entrenched it’s hard to change, but not impossible.
Record and estimate all tech-debt decisions
Intentionally accumulating tech-debt and failing to keep track of it is a sure road to disaster. Unrecorded decisions get forgotten very quickly. Debts don’t go away just because you’ve forgotten about them. It’s vital you ensure your tech-debt decisions go into your project backlog and get estimated along with all your other user-stories and tasks.
Be sure to associate a real business benefit to your tech-debt tickets so your non-technical team members understand why they are important.
Have standardised contributing rules
Every project must have a
README file that outlines what the project does, and how to build, run, test, and deploy it, and that documents any specific knowledge needed by a developer coming fresh to the project. All projects ought to also have a standard
CONTRIBUTING file that outlines how to contribute new code to the project. This file needs to explain that you follow the forked
git-flow process (for example), how to name branches, commits, and pull-requests, and what the definition of ‘done’ actually is.
Adherence to this document must be enforced.
Pull-requests and peer-reviews
Programmers must be forbidden from pushing code directly to
master branches and must always contribute code via a pull-request. Pull-requests must always be reviewed by peers before being merged. Peers must be encouraged to be critical of each other’s code, to nit-pick, and to call out tech-debt when they see it.
Programmers must have a culture of taking such criticism on board without taking it personally, and must have the habit of not merging until their pull-requests are approved by a peer.
All decent programming languages have
linting tools. Use them. Agree up front on the coding standards you are going to use, and then ensure those standards are adhered to.
Ideally your tests cover at least 90% of your code. There is a law of depreciating returns on code-coverage so it’s sometimes not worth shooting for 100% coverage, but if your code-coverage is below 90% you will certainly be introducing problems.
Tests must be regarded as the source-of-truth for the definition of the project, and not as an afterthought.
Clean up as you cook
It’s best to clean up and cook at the same time rather than accumulating a massive pile of dirty dishes. Coding is the same. Try to clean up any tech-debt before you commit your changes, rather than accumulating a massive pile of dirty dishes that quickly become fly-blown and infested with maggots.
Be ‘The Richest Developer In Babylon’
In 1926 George Samuel Clason wrote a book called The Richest Man In Babylon. In it he outlines, by way of parables, a very simple strategy for on-going financial security. The lesson of the book is simply to ensure you save 10% of everything you earn, and avoid going into debt (unless in doing so you can use the money you borrow to generate more money than the interest you pay). Clason also recommends spending 20% of everything you earn in paying down debt, and setting aside a further 10% for ongoing investments.
The lessons are applicable to paying down tech-debt: Devote 10% of your development budget to planning, 10% to on-going developer education, and 20% to addressing your tech-debt.
Invest the time to regularly refactor your code. Doing this properly is only possible if you have decent test coverage. Doing it badly just accrues more tech-debt.
- Small amounts of tech-debt can serve genuine business needs but add medium-term and long-term risks to a project as the debt-pile grows. The risks inherent in taking on tech-debt are impossible to quantify in advance.
- Managing tech-debt is not actually that hard if you get the culture right to start with. However, if you have a culture of sweeping tech-debt under the rug and ignoring it, you are inviting trouble.
- No-one wants to live in a share-house where people don’t clean up after themselves. After they’ve been bitten by rats, and found maggots thriving under the carpet, most reasonable people will just move out.
¹ However it’s also true that ‘tested is better than shipped’. Tests are the ultimate source-of-truth for your project’s specification.
² 80% of statistics that cite the number 80% are just made up. ‘80% of’ is a common cypher for ‘many’. Likewise ‘20% of’ is a cypher for ‘some’, as only 20% of stats that cite the number 20% are likely to have been backed up by actual research.