Spaghetti for dinner? Sure! Spaghetti for code? No thanks
Some tips on how to avoid spaghetti code
For any application, there’s an infinite amount of ways to implement it. When engineering software, one can always follow the best practices and technologies that are meant to help achieve your goals, but the problem is — the best practices and technologies available continuously keep changing, usually making the systems rot.
Even when doing quality research and picking out tools that seem to fit perfectly, after some time (hopefully at least a few years) the inevitable usually happens — the software that was built with care, by the best engineers, using the best technologies at the time, will become outdated and (more importantly) even cumbersome to work on. Assuming the most important principles of software development: such as the single responsibility principle, DRY (don’t repeat yourself), KISS (keep it simple), etc have been consistently applied, the resulting code probably can’t be referred to as a “spaghetti”, but nevertheless, something along the way has happened that made the outcome not perfect either.
Why does this even matter?
Software systems in this kind of outdated state are harmful because they become difficult to tweak and slow down the overall forward progress. That said, the importance of these outdated systems shouldn’t be underestimated— many of the older systems our world relies on, have these shortcomings, but yet they do their job and thus are still useful.
Pipedrive’s codebase is split up into hundreds of microservices, with some amazing engineers with a strong drive working on them, but we still do have a few services that follow the less favourable pattern described above.
So, what ends up building a legacy codebase that is hard to work on? I believe that answering this question is crucial and leads to practices that help avoid it happening again in the future.
How did we get here?
Much like a recipe will guide you through how to complete a meal (spaghetti), here I will list the different steps that led us into what I would consider some rotten code.
1. Agile development
Although agile software development is considered the “holy grail” nowadays (and I do tend to agree with the approach), writing software in an agile way puts most of the focus on the short-term plan, sometimes resulting in a working product that is backed by a spaghetti codebase.
In these situations, the next iteration may fix some of the previous issues but more often than not, adds even more.
2. Premature optimization
One would think that trying to come up with a cleaner structure for code is exactly what should be done to avoid a codebase that is hard to work on, but I must say, I’ve seen several times where it turns out the opposite. Striving for a cleaner code helps avoid spaghetti, but might create a lasagne or a micro-service hell.
Not only can code with a long stack trace be hard to follow, but it’s also hard to find the right layer to implement the change in, and the rudimentary parts of the code that are not widely used nor developed, start to rot.
3. The wise passiveness
A big part of the hard-to-work-on code is created by the mantra spoken a lot within software development … 🥁 …: “if it’s not broken, don’t fix it.”
While this approach makes perfect sense when the aim is to keep the systems stable, it is often also the reason why code is not refactored, migrations to newer technologies are not conducted, and dependencies are not updated. All of this could ultimately lead to a situation where the codebase incorporates — hacky solutions, is really hard to refactor, or is tightly coupled with an ancient dependency that could not be updated.
The risks an update introduces often outweighs the gains, but over time the risk might grow into unachievable while the gains become a need.
The ultimate advice
So, what should be done then? Unfortunately, I haven’t really figured that out yet. Maybe it’s inevitable that most long-running systems will end up as legacy. Maybe this is too hard to change and the best thing to do is to accept it.
Although I am unable to foresee the future, here are my [opinion-based] suggestions for making a codebase more “future-proof”.
There is no universal guide nor absolute truth that will help you build every aspect of an application in the best way. The same is true about the proposals stated in this article. It does not intend to fill in missing gaps in processes and the thoughts presented here should not be used for bringing up a fight nor followed blindly.
Make it easy to refactor
Make this a priority whenever writing a line of code. Usually, when doing agile development, the most crucial thing to consider is to build as little as possible to fulfill the needs of the day — not to over-engineer, because the needs may change the next day.
I think this is a good mindset, but would actually push you to think even one step further — if you already know there are going to be lots of (still unknown) changes, make it easy to apply the changes.
The ease of which to refactor a codebase is often treated as a product of the code structure. The fact that different technologies and languages tend to have different refactoring curves, is often overlooked. What does this mean in practice?
Luckily I have a very scientific (and professional) graph for it:
Nowadays, many developers dislike statically typed languages, and I share some of their concerns about added boilerplate and type system limiting the development speed. On the other hand, moving around and renaming files, functions, and variables in a statically typed language (i.e Java), with the help of the compiler and an IDE, can be done without any fear of breaking the functionality. Even though IDEs have evolved, the same thing can’t be said for dynamically typed languages.
Instead of trying to work out the ideal directory structure of a repository, make it easier to alter the structure in the future. This can be achieved by using technology and code structuring practices that support the idea. Although neither are actually statically typed, you could still consider it one of the benefits of Typescript over Javascript.
To be clear, I’m definitely not stating that statically typed languages are superior, but rather that they have their advantages and disadvantages. It all depends on the application, team, and how it is set up.
Reason to write tests
I have heard many times in my life people say that tests are useless and just slow down the progress. Almost everyone arrives at the thought “why should I write code and then write another trivial code to run it, totally unnecessary”.
I myself have definitely sometimes felt the annoyance of producing tests.
Of course, the need to apply changes in the future almost always justifies writing tests today. Unit tests are useful for validating smaller structural changes and functional tests for validating bigger refactorings. Refactoring a code that has no tests is like perfectly following your New Year’s resolutions — you’re not likely to do it.
The biggest benefit to having quality tests is not that the tests validate the behaviour of the code, but the fact that tests enable future refactors 💡 — by validating the behaviour.
Bring in new people
The truth about the quality and readability of code only really comes to light when involving engineers that have not worked on that codebase before. Everything, even if it shouldn’t, starts to make sense when stared at and tweaked over a long period of time. But the one looking at the code the first time tend to have a different view, and might not be satisfied with the solutions.
Needless to add, but newcomers benefit and improve the codebase by bringing in new ideas and applying different thinking and skillsets.
Having “new people” does not need to mean constant hiring, it could be easily organized by just rotating some engineers, who are willing to step out from their comfort zones and move between projects. In Pipedrive, the mission framework in use has served us well for this.
Although the article could certainly benefit from it, I unfortunately do not have a graph for this so I instead used my artistic skills and drew this picture to help you visualize:
Juniors!
The more experienced a developer is, the better they are at understanding a complicated implementation — but there’s rarely ever a good reason to stick with a very complicated implementation. The extra 10 layers of abstraction and generalization are not needed. The smart and cool oneliner that makes the code harder to read looks nice, but ultimately must be dropped. Keep it lean, easy to follow, read, and edit.
If the code is too hard to understand for a junior, it has to be made simpler. This simple concept makes juniors a very important part of any project. Let them review the code! The lack of experience they have might sometimes be unexpectedly useful.
Update dependencies
Make it a routine to update dependencies and frameworks. The moment you feel the functional need for an update, it may be too late to migrate your service to the new update. Keep an eye on how your dependencies evolve, i.e by subscribing to their release announcement lists.
Recently, multiple tools have started to offer updating dependencies automatically, usually bound to the security issues discovered in libraries. This must be a step in the right direction, but do not stop there.
Trigger version checks even when there isn’t a known security issue. Each update still requires a dedicated look into the changelog, do not apply an update to the dependencies blindly. I would also recommend not rushing to update dependencies to the latest versions right after a release — there usually is a great flow of incoming bugs when something is newly released.
Adopt new technologies
There is a constant hype on everything new and it is usually not wise to be the first one changing the technological stack. Taking advantage of anything good that new technologies and techniques bring will need a shift in existing systems.
Many of the ideas evolving in the software development world are, in my opinion, related to the same topic — how to avoid creating a legacy. It is useful to keep an eye on the new stuff. Many of the new ideas can be successfully ported into the existing stack.
Also, question the decisions made in the codebase regularly. It could be that something that looked good and made sense some time ago, no longer does. Not long ago, following object-oriented paradigms seemed to be the best approach, but nowadays many would agree that extending a class is limiting and the possibility of having a method overridden only creates confusion.
Split it
Last but not least, keep the separable pieces of your application small. This can be achieved by just keeping your code less coupled or maybe by using microservices. There might be other benefits, but the main reason to split the application into smaller independent chunks is that smaller chunks tend to be easier to digest and enables the possibility to rewrite parts.
Conclusion
There are many useful principles for doing software development. Keep on using every one of them as long as they make sense and serve a purpose. Simultaneously, try to revisit these ideas, as your codebase grows or technology around changes, they might not make as much sense as they did.
It is more important to make code that is well refactorable than it is to write the best possible code for the first deploy.
Tests enable refactoring capabilities so try not to forget to make use of these capabilities: take time to update the dependencies and adopt new technologies. This will keep your application safe and cool. Again, bring in new people and listen to what they say — that is the best source for new ideas. Lastly, attempt to refactor the codebase whenever you see the need.
Adapt to new before it gets stale, fix it before it gets broken. Have fun!
I am eager to read what your thoughts are on why code get rotten over time in the comments and what you agree with or disagree with in the article.
