TL;DR: Writing software is hard. Technical debt is a huge piece of what makes it so tough. The Mito engineering team has lots of experience writing code, but we’re still improving how we manage our debt. Here are some rules we use stay out of a debtors prison.
Mito’s Engineering Team
At Mito, we’re building a spreadsheet that records (and replays) Python code while you edit. This is no easy task — so it’s a good thing we’ve got a crack engineering team to accomplish it.
Our engineering team is made up of two folks: myself, and Aaron Diamond-Reivich. My background in software is working on some open-source projects (that no one ever used but me), and doing research (that no one ever used but me). They were cool projects, but a far cry from production code. Aaron’s background is in data analysis — and while he’s a great programmer, he also has little experience maintaining large-scale production systems.
Simply put, the Mito engineering team is still figuring out how we like to write and maintain production software — so imagine our horror as we first began to encounter technical debt we made for ourselves.
The Time Before
The first lines of code you write are easy. There’s no constraints from what you’ve done before, as you’ve done nothing before. Sure, you’ve got the constraints of your product spec (details on this process coming in a future blog post!), but you’re not fighting your past (idiot) self to write any lines of code. It’s just you and your current (idiot) self.
In our case, this dynamic of “no-constraints from previous code” changed right around week 4 of development.
Mito is a spreadsheet that generates Python code for each edit you make. At a high-level, each edit you make is treated as a “step,” and each “step” is then turned into corresponding Python code. For example, when a user writes a new spreadsheet formula, this generates a set_column_formula step, and a set_column_formula step then converts itself into valid Python code. Woo woo!
However, we initially designed this step generation pipeline wrong in a few ways.
First, adding a new step type required “registering” the step in about 5 places in our codebase. In practice, this just meant you always forgot to register the step somewhere.
Moreover, adding a step required adding a data structure to store the parameters and effect of the step. In turn, all already-existing step had to worry about and maintain these new data structures. This meant adding a new step required changing all the existing steps. Again, in practice, this just meant we forgot to make these changes.
So, gradually, we re-architected. We created a single place where steps could be specified, and then automatically registered them in the 5 places they needed to exist. We also standardized the way steps store their parameters and effects, making it much easier to add steps without breaking already-existing ones.
This was the first major example of technical debt in our codebase that we had to pay down — but certainly has not been the last. As the workflows we support get more diverse, and our codebase grows, so too must we continue to manage our technical debt.
How We Learned To Stop Worrying and Manage Our Technical Debt
We’ve begun to flesh out a few rules for technical debt management that guide how we think about it. These aren’t hard and fast rules, and many of them we stole from other technical blog posts of HN, but they have helped us make decisions in a pinch!
Your Codebase is Your Workspace
Just like U.S. Treasury bonds, there are two types of technical debt. Long-term debt is architectural: incorrect structural decisions that turn out haunt your days. Short term debt is code-level: breaking your functions up wrong, or bad variable names, or just messy code generally.
Long-term debt is hard to escape, and expensive to fix up. Architectural decisions, obviously, effect much of your codebase, and so changing them takes a while (and a high-bar of pain) to motivate one to fix up.
Short-term debt is a bit easier to clean up. In fact, we generally try and eliminate all short-term debt before it makes it into deployed code. We’ve discovered there are two strategies for eliminating this short-term debt:
- Implement a new feature. Open up a PR. Clean up the PR (and pay down your short-term debt) before requesting review.
- Implement a new feature. Clean as you go to minimize the amount of short-term debt in the codebase at any time. Finally, open up a PR and request a review.
We recently decided to explore the effect these different styles have, and taking a note out of every highschool-football star’s playbook, recorded ourselves programming.
In watching the footage back, it became clear that waiting to clean your code left the programmer with dirty code the whole time they were programming. In practice, this dirty code led to more bugs that blocked the new feature’s implementation. If you instead cleaned while you went, you could avoid most of these silly bugs, and implement the new feature much quicker.
TL;DR: You codebase is your workspace. Short-term debt should be paid as soon as possible, so you don’t have to pay any interest. Clean as you go, not at the end.
Pay the bills with the highest interest rate first.
There’s lots of technical debt in any codebase. We’ve refactored large portions of our codebase a couple of times, and will continue to do so. But what to refactor?
Simply put, we refactor the thing that we waste the most time on working around. This seems obvious, but usually the most-painful and as-of-yet-unrefactored code is the code that will be most painful to refactor (for obvious reasons). However, refactoring this code can lead to the largest productivity gains long-term — as was the case when we refactored how we handled steps.
Fear not! Go forth into the technical debt and bad decisions of a days-past and bravely vanquish that debt that causes you the most anguish currently! For if you destroy that foe, what left can stop you from building the product of your dreams?
Just kidding. But seriously. Don’t be a knave.
TL;DR: Refactor those things which cause you the most pain first.
Don’t pay bills until the debt collector knocks on your door (or you’re reasonably sure she will).
This point is highly related to the one above, and is most easily expressible with: we don’t refactor debt that looks ugly but hasn’t hurt us yet.
There is super-visible debt, and there is debt that is less-visible but still effects much of the code you write. Instead of cleaning up the eyesores, spend time thinking about where your technical debt actually slows you down — and fix that up instead!
In our codebase, there are a couple of graph algorithms we wrote a few months ago that don’t currently meet our ever-increasing standards for cleanliness. But they’re well tested, and not buggy, and not causing issues. So we’re not touching them for now.
TL;DR: Visibility ≠ importance. To quote the late, great David Graeber: not all debts need be paid.
As a consumer with changing preferences, being in debt is inevitable.
Some of the technical debt in our codebase is due to our own bad decisions: two functions that are tightly coupled and badly composed, or a system for registration that requires registering the same thing in 5 places.
But other pieces of technical debt are not from our bad decisions, but rather from our changing requirements and evolving understanding of our users. We originally architected our system to work just on a server — because we thought this would be the easiest solution for our users. Turns out, some folks really do prefer keeping their data local. So comes the debt.
These changing requirements and understanding will never go away, at least as long as our company exists. And so as we grow into our roles as software engineers, we must learn to live with technical debt in Mito.
Try Our Debt-Managed Software
If you liked any of the above rules, thanks! If you have others that would help us, leave a comment or shoot me a tweet — we’re actively looking to learn!
The above rules (we think) result in pretty good software. If you’re interest in checking it out, try Mito.
