How To Deal With Technical Debt

Technical debt can cause so much frustration and burnout to development teams. Software engineers can be aware of the side effects of technical debt. However, they often need to explain to the product team why quick and easy solutions to coding development are risky.

The term was coined by Ward Cunningham in 1992. at the OOPSLA conference [1] as a metaphor for developing a software asset. He concluded that the development process leads to new learning, as it depends upon artifacts he coined as technical debt.

For Technical Debt, we can say that it’s everything that stops us from developing software fast. Martin Fowler explains in his technical debt quadrant four different pathways that lead to technical debt creation [4], but there are more. And we can see this, especially with startup companies who want to go fast and don’t think about quality first.

In studies on developer productivity [2], developers are usually forced to introduce a new technical debt as companies keep trading quality for short-term goals, such as new shiny features. Yet Technical Debt doesn’t only impact the whole organization but also developers’ happiness, job satisfaction [3], and morale [8].

The main problem with Technical Debt is that code is an abstract concept, so it’s hard to explain what happens to business and management. So, companies could easily ignore what they don’t see or understand. So, here, we need to be explicit and visualize our Technical Debt clearly for the company and management.

Usually, during the development of our software systems, the capacity available for new features per development cycle decreases (lead time is increasing) while complexity builds up. So, we spend more time fighting the complexity than developing new features.

The image below shows how technical debt reduces teams’ ability to develop new features.

Over time, the capacity to develop new features decreases

We want a high-performing organization that will give us a competitive advantage to be more responsive and innovative. We need a high-quality software development process to do this quickly.

Technical Debt will occur anyway, yet we need a strategy to reduce the creation of new debt while allowing us to reduce existing debt. Investing in this will build technical wealth for our company.

Types Of Technical Debt

We usually think bad code = Technical Debt, but there is more to it. There are different types of tech debt:

1. Code quality — our code could be clearer to understand. There are no coding standards, it’s poorly designed, and there is a lot of complexity. In addition, there are a lot of code comments explaining what something does.
2. Testing — we need a proper testing approach (check Test Pyramid), from unit tests to integration to E2E tests.
3. Coupling — between modules that block each other increases the time to deal with such code.
4. Out-of-date Libraries or Tools — we are using some legacy libraries or tools with several issues, such as security or inability to update to new technologies and platforms. We, as developers, always want to work with the most up-to-date technologies and efficient tools.
5. Manual Process — some processes in our delivery need to be automated. No automated builds, no CI/CD process.
6. Wrong or no architecture — we don’t have proper architecture, or it’s just a big ball of mud. Our architecture doesn’t reflect what we want to achieve with our system, or it doesn’t scale well.
7. Lack of Documentation — there is no documentation, or it’s not updated to reflect the system’s current state.
8. Lack of Knowledge Sharing — we don’t have a culture of knowledge sharing, so it isn’t easy for newcomers to get up to speed. We should always document our decisions and specifications during our work.

And there are more types, too, regarding low-value work, issues with monitoring, etc.

Yet, we often see in many organizations a need for more processes and strategies to manage and reduce technical debt, which is a sign of needing a proper engineering culture.

No time for improvements

Strategies to Fight Technical Debt

So, what are some strategies to reduce technical debt? Let's start by prioritizing Technical Debt. First, we must analyze our development process and codebase to find bottlenecks and create a Tech Debt Map. Then, once we have such a map (shown in the image below), we need to review each piece by identifying causes.

When we have this list, we can go through it and check what happens if we do nothing (could it become worse or not) and if this part of our system is used for new development now and in the future. So when we mark these two things, we can prioritize and keep stuff with worse tech debt, and we will build on those now and in the future.

Also, another alternative is to build this map in terms of effort and pain to fix issues or the RICE Scoring model.

Tech Debt Map

You can also organize a workshop or Tech Debt retros to build a map, and you should regularly audit debt across all of these categories.

A Recommended Strategy To Deal With Technical Debt

One recommended strategy to deal with Technical Debt consists of the following steps:

1. We need to be transparent about our development, bottlenecks, and why we need to go faster. Then, we need to summarize this in a language that allows all stakeholders (engineering and management) to share the same understanding. Then, Here, visualizations help a lot.
2. Our systems need to have clear ownership and be responsible entirely for them. But, unfortunately, if some parts of our system aren’t right, no one is taking care of them.
3. We need to prioritize stuff based on impact, as we showed on the Tech Debt Map above.
4. We must empower teams to fix problems and resolve Technical Debt in the natural flow of product development. We need a healthy balance between reducing tech debt and adding new functionality with a pragmatic approach. What works well is to continuously invest 10–20% of development time into this. How you can do this:
5. For smaller debts (up to a few hours), do a refactoring when you touch them. But, then, let people be Good Boy Scouts.
6. For medium debt (from a day to a few days of work), you can try the following: Tech Debt Fridays are working only on this, or mark tech debt as a feature, prioritize it, and work on it.
7. For more significant debt (from a few weeks to a few months), take fixed time for tech debt. For example, take a story and work two days on it, or have a special task force (dedicated team) handle it for some time.
8. Use metrics, such as code issues (e.g., by using SonarQube) or lead times. These metrics can help us make better decisions about fixing the technical debt.

Tools to track Technical Debt

There are different kinds of tools where you can track Technical Debt. However, I recommend always using some existing tools because introducing new ones can be costly. In general, Technical Debt can be followed in the following ways:

1. Project management tools — the tools such as Jira, Asana, GitHub issues, Azure DevOps Board, or some more specific tools for this, such as Stepsize, CodeScene, or NDepend (for .NET).
2. Automated code analysis tools — you can use tools for code analysis, such as SonarQube, to track and detect code issues, security issues, or other problems.
3. Manual tracking method — here, you can use your manual way, which most fits you, such as Excel table or whatever works for you. It can even be a combination of the above methods, such as tracking strategic Technical Debt in Miro, tracking tactical Debt in Jira, and using metrics from SonarQube. Just make it accessible and easy to update.

CodeScene for tracking Technical Debt (Credits: Adam Tornhill/CodeScene)

Keeping Technical Debt Low

We want to produce code with the right speed, quality, and complexity, not in a quick & dirty way. As software evolves, our systems' architecture becomes more complex, making maintenance difficult. Yet if we did a poor design and code quality from the start, we would be in trouble later.

Our goal is to produce systems that are easy to maintain, that we can avoid errors, and that issues and further improvements can be added at a low cost. Software architecture and craftsmanship should be appropriately set to achieve this. We want to balance quality, speed, and complexity (as shown in the image below).

Yet, the faster we develop new functionalities, the more it is to ensure software quality, and the tech debt tends to pile up, leading to a slower development process and more errors. So we want to go slower from the start, which will increase our speed in the future.

The ideal situation for software development

The question is how to avoid creating tech debt in the first place. The answer is to have a high-quality software development process. Here are some things we can do:

1. Architect systems for the desired outcome. Involve architects from the start in the creation of teams by defining their topology [7] because otherwise, Conway’s Law [5][6] will make our architecture an image of our organization’s communication channels, not the desired architecture.
2. Write documentation for all your architectural decisions (ADRs) and technical documentation. Make it easy to find (place it in the repo near the code).
3. Write tests on all levels of the Testing Pyramid. Aim at 60–80% of code coverage.
4. Encourage a code review culture. Involve everyone in code reviews and check for code readability, design, performance, security, testability, and documentation. It is one of the best ways to share knowledge.
5. Pair Programming works where one developer writes an ode while another provides feedback and suggestions. This results in higher code quality; we don’t need code reviews with this implemented.
6. Allow developers to refactor. We should build such a culture that there is no need to ask permission to fix technical debt! Especially if it’s something that can be done in a few hours.
7. Follow codebase standards, such as design, naming, and architecture. Then, automatize this through tools such as linters or your CI/CD process.
8. TDD (Test-Driven Development) is a software design process where each functionality is designed by writing a test first and then doing a proper implementation.
9. Write high-quality code. Educate your developers first to know their programming language in-depth and understand OO concepts, design patterns, refactoring, clean code principles, architecture patterns and styles, and SOLID, YAGNI, KISS, and DRY principles.
10. Continuous Delivery. Code is adjusted in short cycles, so it’s always easy to verify by other team members. This method avoids significant issues in production by employing frequent releases and fast feedback.

References

[1] “The WyCash Portfolio Management System.” Ward Cunningham, Addendum to the Proceedings of OOPSLA 1992. ACM, 1992.

[2] “ Software Developer Productivity Loss Due to Technical Debt, “Besker, T., Martini, A., Bosch, J. (2019).

[3] “ Happiness and the Productivity of Software Engineers,” Daniel Graziotin, Fabian Fagerholm, 2019.

[4] “ Technical Debt Quadrant,” Martin Fowler, 2009.

[5] “Conway’s Law,” https://en.wikipedia.org/wiki/Conway%27s_law.

[6] “Conway’s Law,” https://martinfowler.com/bliki/ConwaysLaw.html.

[7] “ Team Topologies: Organizing Business and Technology Teams for Fast Flow,” Matthew Skelton, Manuel Pais, 2019.

[8] “ The influence of Technical Debt on software developer morale,” Terese Besker, Hadi Ghanbari, Antonio Martini, Jan Bosch, Journal of Systems and Software, Elsevier, 2020.

Read the rest of the article, including bonus chapters in the Tech World With Milan Newsletter.

To expand your knowledge and personal growth, subscribe to my free weekly newsletter with 20,000+ people: https://newsletter.techworld-with-milan.com.

Originally published at https://newsletter.techworld-with-milan.com on March 23, 2023.