How to manage tech debt? We did a tech debt week and it was a success
Tech debt in plain words is a shortcut or a known suboptimal choice made to expedite the delivery of a software product. The shortcut can be in code, architecture, or other factors. If it is an informed decision, generally a “commitment’ is made to refactor/rewrite the thing to pay off the debt in the future. Does it really happen is another question to ponder upon. One argument is to write the code without cutting the corners but that is not possible all the time.
Similar to financial debt, tech debt also has the advantage of moving faster and if too much debt is taken then you will get buried under interest you have to pay. For financial debt, the interest is money and for tech debt, the interest is slower velocity, hard-to-understand codebase, and even higher attrition rate among software engineers who have to deal with these difficult-to-work-with codebases. At Simply Wall St. we conducted our first tech debt week at the start of January this year to work on tech debt for a whole week, here is a rundown of it.
Projects
We use confluence to list any ideas and there were pages to list out tech debt issues segregated by Backend, Frontend, and data issues.
Some of the projects that either had very good progress or got done were:
PHP static analyzer
Even though our newer applications are written in Node.js (and TypeScirpt) for the backend, there are some older applications that are in PHP. Along the same lines, PHPStan was implemented in a couple of projects to run static analysis on the PHP code. These projects now have PHPstan running as part of the CI/CD process and check for formatting issues and errors at an appropriate level.
After adding PHPStan some of the code has already been updated to be more standard-compliant and less error-prone. There were useful suggestions from PHPStan when ran on the codebases and relevant ones were implemented and tested by the involved engineers.
Updating Node.js for projects
We use GraphQL as our primary communication medium between new services these days. All of our newer services are implemented in Node.js and our framework of choice is Nest.js with TypeScript. Our GraphQL adoption journey started a couple of years ago and we have over 50 microservices to do different tasks for our platform.
For the tech debt week, a small team of engineers migrated 40+ services from Node 14 or 16 to the latest Node.js LTS version 18. There were some libraries that had to be upgraded in the process. The engineers also added DataDog APM to a couple of services while upgrading Node.js. Compared to upgrading PHP applications, upgrading the Node.js applications was a much smoother and easier process.
Move analysis to another technology
Some members of the data team were able to move the company analysis tasks from an older technology to a newer one. The analysis was earlier done with a different technology and the engineering team was responsible for all the calculations.
Thanks to the tech debt week, the data team go the time to move all the calculations to a newer technology and also take ownership of the process. Similar to the earlier solution the events are propagated to other applications using Rabbit MQ. Other systems listen to the relevant events and updated the data related to the event emitted by the updated system to keep important data synced across data stores. There was a good deal of collaboration between the engineering and data teams to get this task over the line.
Remove unused queues
We use RabbitMQ as our message broker of choice to enable an event-driven architecture where multiple microservices communicate with each other using queues. With time the number of microservices has grown resulting in more queues. For the tech debt week, one of the tasks was to delete unused queues.
Not only did the unused queues too up unnecessary resources, but they also caused confusion when debugging for issues. For these tasks, some unused queues were removed, and some of them also stemmed out of the above migration for the analysis calculations. Overall the health of the queues in our RabbitMQ server looks much better and there are no queues now that don’t have a consumer listening to them.
Update dev dependencies
This was mainly led by the frontend team, the team was able to update some important npm packages in some of the projects. The dev dependencies that got an upgrade include webpack, babel, eslint, and TypeScript. Upgrading these packages has already made the developer experience better by making the builds a bit faster and catching any out-of-standard code at the development stage than a later stage.
Other Avenues
At Simply Wall St. we also do quarterly hackathons and sometimes the teams do projects that pay off some tech debt. If you like what you have read so far, you will love 5 compelling reasons to work for Simply Wall St. tech too. In 2022, we even migrated our infrastructure and databases to AWS.
Even though allocating a set amount of time to pay tech debt is a good idea, the better idea is to pay tech debt continuously. A practical way to do it is to follow the Boy/Girl Scout rule — Always leave the code better than you found it.
Conclusion
Thereby, our first tech debt week was a success. The above projects were either completely done or completed to a great extent. There were other bigger pieces of work that had the base laid and needed more work to be done.
A company investing time to particularly pay off tech debt is outstanding and we as a team really value this opportunity.
Similar to financial debt, if the interest and installments including the principal are not paid on time that will result in severe issues. The same thing happens for tech debt, accumulating too much tech debt is equivalent to signing a suicide note, it is just a matter of time before everything comes crumbling down. Take up the tech debt that you can pay in near future reasonably. Events like tech debt week are a good avenue to pay larger tech debt items, still, a culture of delivering smaller debt improvements continuously will go a long way.
In our case, the impact was felt immediately after the tech debt week. The developer experience for the frontend team was better, and for the backend and platforms teams, there were lesser patches to worry about as all Node.js versions were upgraded to the latest LTS. Similarly, the data analysis pipeline was smoother too.
Kudos to the idea and the execution, we are all looking forward to the next tech debt week at Simply Wall St.
