Fancy Building a Framework?
What is your reaction to the trigger phrase “opinionated” framework.
Your opinion versus mine!
This topic is about the realities of defining architectural patterns for large systems, in actual code (which is common need in a software businesses where you are trying to gain some efficiencies in reusing learned patterns and practices across many software products/projects).
This will be a familiar experience to many:
We’ve all seen them in their various forms, those cool frameworks, and some of us have used a bunch of them extensively over the years, so we have come to learn about their esoterics, intricacies, limitations and constraints that they impose on us over the months to years of persisting with them. They are especially spicy when the honeymoon is over and we outgrow them, but can’t rid ourselves of them once the become unfaithful to us. Ultimately, there are very few good ones out there, and even fewer that endure for very long.
Many of us have butt right up to their limits and even had to workaround them (with explicatives). It is a common issue that most programmers encounter at some point.
We love them, we hate them.
Some of us have even gone to bat with the authors of those frameworks pleading with them to make our specific use-case important enough to consider to change their framework to accommodate it. Many have failed at that pursuit (in a lot of cases, quite rightly failed to change the framework, as it was never intended to deal with their use case). Some, skillfully succeed at expanding them, and the world is a better place.
Alas. It is a general problem of adopting 3rd party frameworks (open source or proprietary), that seems to persist through the decades.
If you persist in this career, you might also go on to view each one of these attempts (at building a useful framework) as a laudable failure, and boy! there’s certainly plenty of no appreciation and no shortage of the grief some programmers can give about the pain they experience using them.
Some programmers even see their own experience and pain with the frameworks they use as indicative of a general problem that needs solving for all programmers across the industry. In the naïve pursuit of some kind of holy grail for the perfect way to do things. For some, that crusade turns into a personal challenge to them to define the ultimate framework to address this pain, that surely will vanquish that injustice, and as a side-effect, they will be hailed aloft a hero. And alas, that’s where the real learning begins for them.
All models are wrong, some are useful
The truth is, that no matter what mental model you have of whatever problem you look at in programming, the moment you commit to writing some code to resolve that problem, you are going to have to make some compromises in your implementation of that model. Why is that? Why can’t we have nice things?
A. Because of “why” all models are wrong!
A model is simply an abstraction across a set of problems, that helps you neatly understand and describe those problems. That model cannot accurately and precisely define the fine details any one of those problems, because that would require an infinite level of fidelity in your model, which would converge ultimately to just describing reality (i.e. no abstraction). So, if you want a model to simplify a problem, expect that it won’t capture all the fine details. This can be useful in a lot of cases. But remember, in programming:
The devil is always in the details
So, any abstraction you create will necessarily have to trade certain things off to achieve usefulness, and thus high usability.
This takes some learning, let me tell you. Having started to build large scale architectural frameworks for enterprises back in the early 2000’s, I very quickly learned that no matter how diligent I was, no matter how clever my abstractions were, no matter how many use cases I tested it on, there were always edge cases or use cases that I couldn't elegantly deal with in my abstractions. Yeah sure, you can create extensibility points to accommodate them, and in many cases, you do just that. But no matter how much you refine your abstractions, they are never quite good enough or pure enough.
Then, there are the abstractions that your framework has, that have to necessarily compromise your core design principles. These are the real killers, because often is the case that one of the purposes of your framework is to mitigate some kind of specific complexity/coupling aspect, and there you go obviously violating that specific coupling aspect or adding more accidental complexity to the solution to try and achieve that gracefully in your framework.
Of course, there is always a solution to any problem in software.
Create a better abstraction.
We can create whatever abstraction we want in software, its just a matter of how many layers of indirection you want to tolerate. To those who have to use your framework, you probably just created an unusable monster, that no human can understand or wants to understand. Far too complex to be useful.
So what to do about it?
Well, if you never built one before, you probably just have no idea how hard this kind of activity is, so its unlikely you’ll empathize with those that do this kind of thing. You will likely continue to go on to think that you can always do better yourself, and maybe you could, or maybe you wouldn’t.
But let’s not get off track, and turn this into “who is the smartest person in the room” type of solution. Instead, let’s find a far simpler pragmatic solution instead.
The real issues with reusing frameworks come because of a mismatch between the assumptions in the framework and the actual contexts of the consumers of the framework. If those two things were always well-aligned in each context, then the framework would always be a perfect fit, in every case.
So how do we make that happen?
Only one way really.
Lets build that framework into the code and context of your own codebase.
- You have all the code you need. Its right there, you own it, you can change it. You can easily debug it, and can see how it works.
- If you ever need to understand what it does and how, its right there in your codebase! no funky debug symbols to setup and no disassembly to pick through.
- If its broke, you can fix it. No waiting for some project maintainer to do that for you, and no merge request! pleading for help! Just do it yourself.
- If it doesn’t fit your context, then make it fit your context precisely. It will evolve as your code that consumes it evolves. Never any mismatch.
But there is one MAJOR caveat.
Never, never, never put the framework code in some reusable package (like a set of nugets/npms) and version it so that you can share it, and gain some shared benefit across other contexts. Why not?
Well, if your intent is to share the framework with other codebase contexts, then you just inadvertently mandated that some other poor sole now became the project maintainer of that framework — and you are back to square one. (Not fitting your context precisely, and lobbying them to change it for your context). You see?
Can you put the code in a re-useable package just for your context? Yes of course you can. And you may want to do that. But expect that versioning it and maintaining a version of it for your context will become a giant PITA for you, and lead you down the ugly path of lock-step deployments. But, it does mean you don’t have to recompile it with your codebase every time — if that’s somehow and important optimization in your context.
Building frameworks for reuse in other contexts is hard. Easy to provide useful abstractions that save programmers time. Hard to deal with all their different contexts. Making your framework the bottleneck and blocker to them changing your framework to fit their specific context. They might outgrow your framework, and they will hate you for that.
Building frameworks to be useable is hard. Hard to maintain clean abstractions (in actual code) when productivity is the goal. You are going to have to make compromises, tradeoffs and apply new constraints to be effective, and to create usable abstractions that humans will enjoy using. Do that badly, they will hate you for it.
Building frameworks is easy. Easy to maintain them in your own codebase and in your own context for your own context. But resisting the urge to distribute them and reuse them across other contexts is hard again. So, just don’t do it.