The Life and Death of Large Systems
Once a big development project has gotten up and running, its own weight becomes part of the problem. It becomes a trade-off between taking the time to extend the original codebase, or saving time by throwing in disconnected pieces.
Tracy Kidder described that latter approach four decades ago as ‘put a bag on the side’ in her seminal book The Soul of a New Machine; it is a fitting description of how many developers approach their contributions. They don’t want to get entangled in the earlier decisions that will slow them down. They just want to code.
A classic situation is when the initial team used a technology like Hibernate as an ORM to encapsulate the persistence. It’s a difficult and prickly technology that requires a lot of knowledge to navigate its deficiencies. If it is well-understood, and covers the full width of the persistence base wisely, it can be leveraged for both quality and consistency. But if it was badly integrated, then the incoming developers have a tough choice to make. Spending the time to fix the implementation may not be viable, but just ignoring it and using a different technology for the newer parts of the schema is destructive.
This trade-off can be even worse if the original code is tied to a philosophy like design patterns. When applied as patterns, this concept can really help programmers to implement sophisticated solutions and provide a means of being able to discuss them during the design stage. When applied as a rigid set of building blocks however, it can lead to excessive artificial complexity that has the opposite effect of making the code more opaque. Following suit means spending a lot of time on ineffective boilerplate, but again ignoring the existing code sets a precedence for descending into an Onion architecture.
What to do?
For any large and complex software development, the history of decisions drives both its lifespan and quality. A huge pile of disconnected pieces is unwieldy and at some point such a mess that further work can not be done safely. With that in mind, the obvious long-term approach is to work hard at extending the system, so its death is not premature. For new people joining an existing project, that means getting awareness of what exists, why it exists and how to leverage it. Although that slows down programmer onboarding, it preserves the momentum of the work.
With that in mind, some of the earlier choices in style, convention, or technology were not appropriate for getting the codebase up to the level that is required. So, a big project should be open to changing any of these, it is just that if everyone agrees on making one of these fundamental changes, it is not just applied to any new code, but also needs to be retroactively applied to all of the existing code. That sets a reasonable cost to the choice of switching while ensuring that the work doesn’t die from fragmentation.
