What is Technical Debt?
Recognizing stalled IT projects and opportunities for change
Introduction
The concept of technical debt is often overlooked within software development and IT project management. The annual cost of technical debt (as defined by hours spent debugging or rewriting bad code) is estimated to be about $85B annually. This won’t be surprising to anyone who’s spent countless hours chasing bugs in undocumented spaghetti code.
So, what exactly is technical debt? According to Wikipedia:
Technical debt (also known as design debt or code debt) is a concept in software development that reflects the implied cost of additional rework caused by choosing an easy solution now instead of using a better approach that would take longer.
This pretty much sums it up. It’s almost hard to imagine how something this simple could result in annual losses of $85 billion, but it’s true. At this rate, nearly a trillion dollars are wasted every decade, and it all comes down to the simple choice of doing something easily versus doing it right the first time.
It’s all about choices
Designing and building something in the real world involves a lot of choices, many of which are trade-offs between factors such as cost versus performance, complexity versus reliability, and so forth. The more educated someone is on the finer nuances of the project and the tools of the trade in general, the more likely this person is to make good choices when it comes to solving problems. However, regardless of an individual’s specific education or skill level, this hypothetical person really faces the same choice every time:
- Tackle the issue head-on and get to the bottom of it now
- Avoid choice #1 at all costs (and likely make the problem worse)
Notice that we could easily have created 10 or even 100 alternate choices for #2, but the point is that anything other than solving the problem head-on is not a solution, and therefore any effort spent in that direction is not only most likely a waste of time, but can actually make things worse — much worse. Choice #2 could be any one of an infinite number of excuses to kick the can down the road, blame someone, divert attention, overcompensate, or anything else that doesn’t lead directly to an actual solution to the problem.
The Balancing Act
As stated earlier, developers are constantly faced with challenges, and these can range anywhere from choosing the right features to get the best bang for the buck, to implementing these features in a reasonable amount of time without degrading the reliability of the API and underlying services. To say that it’s a difficult task would be a major understatement, especially when you throw in a bunch of people with different creative input and conflicting ideas of how things should be done. However, the key is to make the best possible choice now. Of course, sometimes that means putting in a temporary solution in place of a permanent one, producing an MVP for a feature in lieu of the final product as a feasibility test, or any combination of quick fixes or updates. All of these are OK — so long as a real, practical solution is planned out.
More often than not, however, making the right choice means evaluating all possible options and selecting one that may require a little more effort or time than the budget allows, or require pushing a deadline back or even possibly considering that the deadline itself may not reflect the reality of the situation. Sometimes, an entire design change is warranted upon the discovery of even the most minute piece of new information, and the longer this is put off, the more of a mess it will be to clean up when that becomes the only option left.
The subtle downward spiral effect
When faced with a difficult situation such as any of those mentioned above, a developer or manager has a tough choice to make: pay the price now, or pay double (or more) later. In some cases, you may get lucky and not have to pay later, such as if an application is due for a rewrite anyway and you can get away with all kinds of hacky stuff in the meantime since you’re just floating the old version along. However, that only applies if your team actually gets around to rewriting the application. If this becomes a habit, sooner or later your entire team is debugging the giant mess you thought you would be replacing, around the clock, with no time whatsoever spent on designing and building the upgraded product. This is a real problem that companies struggle with as they try to migrate away from old, clunky systems to modern services.
More often than not, if you defer some important change, upgrade, or other work item that should be addressed head-on, the amount of rework involved down the road will likely be more than double. Let’s take a look at a contrived example, in which a developer building out a feature runs into a problem:
- Our embattled developer attempts to explain the problem to management and proposes the correct solution. Management insists that feature XYZ be complete and in the client’s hands by the deadline, or there’s hell to pay.
- Developer hacks together something that technically works, but is clearly not very well made, and is prone to strange bugs right off the bat.
- Six months later, the feature suddenly breaks due to some underlying system change which “wasn’t supposed to have broken anything” but was totally incompatible with the hack developed in step #2.
- Developer is now back at step #1, only with the added complexity of reconciling everyone’s pre-existing notions of how it was supposed to work, how it worked last week, should have worked last year, etc.
Clearly, this is not a recipe for long-term success. People often say that they will fix it later or get to it down the road, without realizing that it’s already down the road, right now, and there already isn’t time to fix yesterday’s issues.
Breaking bad habits and inducing positive change
As with any problem, the only way to actually solve it is to face it head-on with no room for escape, no smoke-and-mirrors, and nowhere to pass the blame. It doesn’t matter why it’s broken, who’s fault it is, why language X was chosen for component Y instead of something else, or one of the millions of other ways in which someone could pick the situation apart. What really matters is figuring out how to do it the right way, and then choosing to do it that way.
Let’s look at the case above, but with a more stubborn developer in place:
- Our developer explains the problem to management and proposes the solution as in the last example. Management insists that feature XYZ be complete and in the client’s hands by the deadline, or there’s hell to pay.
- Our developer — recognizing the infinite workload, 85-hour weeks, and blame-shifting that lie ahead — refuses to back down whatsoever and demands that solution ABC is implemented and tested before all else.
- Six months later, everyone has forgotten about the ensuing fight that broke out in the conference room months ago and the team is enjoying pizzas at the launch party for the new version of their awesome software.
- Developer has long since moved on to new features and updates, each of which adds massive value to the system and the company as a whole.
This is obviously a superior track for the company, the developer, clients, customers, family, friends, pets, and everyone else involved in the life of the people who work on this project. Rather than constantly trying to figure out how to mitigate disasters and appease highly frustrated people left and right, this group is delivering solid results which require no explanation or excuses.
Keeping a level head
As we can see, the key ingredient that made the difference between perpetual failure and consistent delivery was the simple refusal to do the wrong thing. Of course it’s easy to lay this out theoretically when one’s job is not on the line, however what ends up happening is that one’s job is always on the line, especially if one is contributing to the overall problem instead of the solution.
So from this we can argue that somewhere in the chain, someone must have enough of a backbone to stand up and flat-out refuse to produce inferior work under any circumstances. As one can easily imagine, this is not always a very popular stance to take, however a good leader will choose to take the right path instead of the popular one, and this is often enough to inspire others in their organization or team to follow suit. In a self-fulfilling prophecy, your team is now rallied behind you and ready to go (in the right direction) and far more likely to outperform expectations in terms of speed and quality.
Identifying realistic compromises that can be made without reducing the overall quality of the product is the holy grail to solving this problem. This is a skill which can be honed with practice as any other skill, and persistence is important. In a nutshell, if you have a choice between putting off the flashy cool new feature versus putting off your test automation development, you have what should be a very obvious and clear choice. However, we all get ahead of ourselves and make mistakes, so take it easy on that guy in the meeting who came up with cool feature Z and can’t wait to get it in the app.
Technical Assets
Facing problems head-on allows teams to shift from the reactive nature of drowning in technical debt, to the proactive stance of creating technical assets which improve productivity, reduce stress, and instill confidence in everyone all around. A technical asset is anything that can be used or referred to at a later time in order to reduce workload, improve the quality of something, or to make any other improvement to the company or any of it’s products.
Here are a few examples of technical assets:
- A process documentation that clearly outlines all requirements for some business process, which serves as a blueprint for managers and developers
- A modern set of software tooling that can be easily replicated and deployed, which allows rapid prototyping and testing of product updates
- A software library that combines many of the most important business functions into a single secure API that is easy to use and maintain
These are a few of the more useful and concrete examples of technical assets. From this we can see how their opposites are clearly problematic for any team, such as having missing or inaccurate documentation or non-standardized ways of building, testing, and deploying new features. Choosing to invest time and effort into these assets will place any team or organization in an excellent position to innovate, solve increasingly bigger challenges with ease, and build excellent products that work properly for years to come with minor upkeep.
Conclusion
While the issue of technical debt is a complex one involving many factors ranging from economic conditions to current trends in technology, the basic concepts apply in every situation. Evaluate your options and narrow them down into the ideal, the good enough, the good enough for now, and the absolutely unacceptable. Don’t try to spray-paint an unacceptable and pass it off as good enough for now, and conversely don’t force an ideal out of a situation where good enough really is good enough. While there are resources available for calculating technical debt, you can get a pretty good picture by asking your own developers, clients, and managers, as these people are almost guaranteed to have plenty of feedback about how things could go better, move faster, run smoother, or be less frustrating in general.
In fact, staying in the loop and actively listening to feedback are two simple ways to make real-time calculations about a team’s overall productivity and whether they believe their efforts are being utilized properly, or if they are being wasted or under-appreciated. Actively listening and accepting the feedback is the critical point here, as the people working in the everyday life of the project are usually those in the best position to offer critically useful and relevant information about how their own work can be improved (of course).
The age-old saying about doing something right vs doing it over not only applies here, it serves as a mathematical formula for the entire issue itself. Save yourself, your team, your clients, and everyone else a lot of difficulty: make a solid effort to discover the right options, and choose to do them.
