There is a concept in software engineering called “technical debt.” It’s when you do something you know is not right, but you are still doing it. As some meme was saying, “we do something not because it’s easy, but because it seemed easy.” So something seemed easy, it took (hopefully) less time to implement it, but now it causes troubles. This is technical debt.
Some project tracking tools consider the output of static analysis tools (such as pmd, findbugs) as “technical debt.” The method is too long, a duplicate string literal, don’t overcomplicate comparison, etc.… Such findings could be useful, though if you have enough experience, such checks are mostly annoying. And it’s obviously nowhere near capturing the real technical debt. There could be a good code with tons of warnings from static analysis tools, and there could be code on the verge of technical bankruptcy, which is doing incredible according to the same tool.
So technical debt is not something easily detected by algorithms. Static analysis tools were for sure developed by smart people, and they would have got it right if it was the case.
Ok, but then what is technical debt?
What is technical debt
It’s clearly possible to measure it, although relative to each other: project “Aurora” has more technical debt than“Bravo,” but both of their debt combined is much less than one of a “Charlie.” In this regard, technical debt is similar to feelings and emotions human beings could experience: it’s hard to say in absolute units how much do you like a particular coffee (or tea), but it’s not hard to give a rough estimate and compare it to other coffee (or tea).
It’s also quite easy for a human to detect technical debt. Usually, you start on a new project and see something bizarre. You ask someone on a project about it, and the reply is, “oh, it’s technical debt. We opened an issue to clean it up a while ago but haven’t worked on it yet.”
It is also something that accumulates. Take a shortcut here and there — fine. A few more corner cuts — it doesn’t look good, but the software does what it’s supposed to do. Though doing the next change, you might find yourself adding a workaround to work around a shortcut you did a while ago. And to actually make it work you need one more little detail “off” when deploying the service... Slowly, but surely, you have to spend more and more time addressing things that have very little to do with the actual change you are trying to do in the system.
And what is especially daunting, is that all of these shortcuts and workarounds are unpredictable and irrational until you get the context. If you are building a CRUD application, key-value store, or a string manipulation library, you can roughly understand what certain component does while looking at the implementation. It’s almost never the case with shortcuts — you usually need a very good understanding of the use case to make sense of why things are done this way.
And it is also inevitable. Most of the time, the software exists to add some value to the user, and sometimes it’s not always possible to have all the resources to deliver this value, not taking shortcuts.
So technical debt is a set of technical decisions and implementation details that result in unnecessary complexity, runtime, development, and operational overhead.
Continuing a financial metaphor, if there is a debt, there should be interest: the time and effort that is spent to fight complications from running and changing code full of shortcuts.
Change after change, it adds up, following the law of compound interest, adding more and more work. At first, it’s more code. Then it’s more operational work: if flag “enable_backoff” is enabled — disable “experimental_fallout” flag, if changing the logic in tracking service — first make sure to update SQL functions in the database, etc.…
The system becomes fragile, requiring more time to address failures, leaving even less time to do the regular development. More quick fixes are needed to resolve urgent problems, which result in more complexity. A specific class of “heroes” appears — people who bravely fight fires in overtime on a weekly basis and save the project from a complete disaster (or significantly reduce blast radius)…
There is nothing bad in the debt
The picture of “technical bankruptcy” when most of the time has to be spent fighting fires doesn’t look good. But the thing with technical debt is that in some cases you don’t have to pay it off.
There is (hopefully) no technical collection agency that would make persistent calls and ask to go and refactor services not to have a shared database. Or to drop the REST protocol and use a proper tool for RPCs to stop writing clients over and over again. Or move some logic to a common place (be careful with this though). In fact, most of the time, management would be happy if technical debt is not touched. It’s frustrating that it takes two weeks to move a button, but where are the guarantees that it would be better if we would spend these two weeks clearing technical debt? Sure, we have to pay the interest today, but maybe we won’t need to touch these parts of the application ever again or would re-write the whole module?
In some cases, this is not as crazy as it sounds. If the interest rate is 10%, but you can get a return of 30%, it makes more sense to have as much debt as possible. A good example of this is a prototype that is thrown away when the main user workflows and system design are more or less established. Or when the success of the product surpasses the planned capacity of the system, it might make sense to keep the system afloat with quick patches while a new system to handle a larger scale is in development.
But even if you are not planning to throw the project away any time soon having some amount of technical debt is good. The example above describes what happens when too little attention is paid to fixing things. But having no technical debt is similar to striving to have 100% test coverage: you have to work very hard to get the last percent and get almost nothing in return.
The key is in the balance.
But how do you know if balance is right? You don’t, it’s impossible. So two strategies are either try to guess when it’s time to pay off some of the debt (essentially, gambling), or to do it methodically.
Imagine that in addition to technical debt, there would be a technical tax for the amount owed. Every year the engineering team has to prepare a form where all taxable events are listed: how many shortcuts were taken, how many hours of operational overhead there was because of these shortcuts, etc. Would it make things work differently in software engineering?