Dependencies of Doom
At Redgate, we’ve built a suite of awesome tools for database deployment. They’re primarily desktop-based and have evolved over time. This is a long-winded way of saying we’re up to our neck in dependencies!
How did we get here? Here’s the story from my memory over the last 5 years or so.
- In the beginning, there was SVN. We treated it essentially as a monorepo and did sparse checkouts so that all code built from source every time.
- We wanted quicker builds, so we moved away from that and conjured up a system to do individual builds and keep our dependencies in sync.
- Along came NuGet, so it made sense to use this internal and publish our libraries to our own package repository.
- GitHub came along, so we moved away from SVN towards individual repositories per product / library.
We want our systems to move faster and at the moment propagating a change from a library is a time-consuming pain-in-the-ass process. It shouldn’t be.
We mapped our systems out. Here’s a tiny fraction of the libraries we have.
At the moment, if you want to upgrade a library at the bottom of the stack it’s a big problem. If it’s a diamond-dependency, then you’re in trouble. Why are diamond dependencies a problem? I.M. Wright puts it like this:
diamond dependencies dramatically decelerate developer dynamism. If the developers of component D want to use a new version of B that depends on a new version of A, they must also wait for a new version of C that works with the new A. The diamond dependency indirectly couples components B and C, creating a bottleneck that slows everyone down. (I.M. Wright)
Note that this isn’t even just for your dependencies. If your library depends on a particular JSON engine then you effectively force that decision onto consumers. his is really important for Redgate. We believe in autonomous teams who can move independently if we have to spend too long coordinating over library updates that’s a bad thing.
It’s easy to agree diamond dependencies are bad, and we’re going to strive to remove them. But how?
It feels like tools should be able to solve the problem, but I’ve not found one. In fact, we’ve made attempts to use tools to solve the problem but always ended up with magic. I don’t like magic, so I think we’ll need some guidelines instead.
- Internalize your dependencies! Don’t expose types your library doesn’t own. Use tools like ILMerge to hide the transitive dependencies of your library (for example, merging in NewtonSoft.JSON)
- Copy and Paste! Sometimes the cost of sharing code is greater than the cost of duplication. We’ll accept that and do that. A recent example was a class pulling in a library just for an Enum. Copying the enum simplified the dependency tree significantly
OK, so fast-forward some amount of time, and Redgate will have rid itself of the evils of diamond-dependencies. What next?
APIs and libraries are still hard. We realised we need to give better guidance to consumers to encourage them to update. We’ll adopt semantic versioning and that’ll improve some aspects, but we’ll also need to develop more of a culture around writing release notes.
Even after documenting those release notes, we still feel our APIs aren’t stable stable and therefore difficult to upgrade. Why’s this? And does it matter?
This is the million dollar question!
I think that’s because they’ve evolved without a clear architectural fitness function. A fitness function is something you can judge an architecture on. SOLID is an example of a fitness function for classes. It’s a bit qualitative but at least it enables you to answer questions about some code and judge whether it’s evolving in the right direction. We currently haven’t got fitness function(s) for our libraries.
What could these be?
- Bob Martin has the Principles of OOD (Release Reuse Equivalency, Common Closure etc).
- Domain Driven Design argues you should organise your libraries around a bounded context. This might mean more textual duplication and more Domain Transfer Objects, but less dependencies across products.
- An eye for the future. Perhaps the measure of success is how easily it could look like a web API? For example, POCOs in and out?
Of course, this could all be rubbish. Maybe we just need a mono repo?
OK, enough rambling. What are we going to do?
- Slowly (but surely!) remove diamond dependencies
- Standardise how we do release notes for our libraries
- Continue the argument about whether using an Enum is a violation of the Open Closed Principle.
