All You Might Really Need is a Monolith Disguised as Microservices
Even after so many advances in software development practices in recent years, the specter of tangled 3-Layered monoliths refuses to go away. In theory, a 3-Layered architecture sounds like a nice and perfect development ideology that talks about separation of concerns, loose coupling and layers, but in practice it quickly spirals downwards into a Big Ball of Mud.
“In theory, there is no difference between theory and practice. In practice, there is.”
Here are some Blunt Facts:
- Until and unless the Architecture itself prevents the developers from taking shortcuts and diluting it, the degradation is bound to happen.
- Until and unless an Architecture is driven by a clear philosophy and doctrine, the team will struggle to make sense of it.
- Until and unless the Architecture is designed to localize complexities, they will spread throughout the system like wildfire.
In practice, the traditional 3-Layered architecture lacks all the above.
Enter — The Microservices
And then came The Microservices and revolutionized our thinking and forced us to think in terms of clear and distinct boundaries for applications. They forced us to think about system not as a whole but as parts. The concept of boundaries is not new. Domain Driven Design has been talking about it since its inception. Microservices showed us a definite way of maintaining the purity of architectural boundaries.
But not all the projects need Microservices and not all companies have the infrastructure, the need, and resources to run a full scale Microservices project.
The point I am trying to make is, if you design a monolith using the mindset of Microservices, you will prevent it from becoming an unmaintainable Big Ball of Mud. In other words, what you need is a Monolith disguised as Microservice.
One important thing to learn from a well designed Microservices is how they maintain their boundaries.
Why are Boundaries so Important?
If you will take away only one thing from this article, take this: Technologies will come and go, but the skill for creating boundaries will always give you an edge in Architecting software systems.
Boundaries Provide Resilience
“That which does not kill us makes us stronger.”
— Friedrich Nietzsche
Take a huge stone (Aka a Monolith) and drop it from the top of a building on a hard surface on the ground. Repeat the same with a loosely packed bag of pebbles (Aka Microservices). Which is more likely to survive the fall without breaking?
Software systems are constantly under stress of Changing Market Behavior, Changing Customer Behavior, Weather (read AWS region outages), Business Course Corrections, Leadership Changes, Technology Changes ect…..
We must design systems to survive stress, otherwise they become extinct.
It just takes a single Black Swan event to completely dissamate the poorly designed monolith, whereas a system designed with proper boundaries has better resistance to Black Swans.
Boundaries Makes Systems Antifragile
In his groundbreaking book “Antifragile”, Nassim Nicholas Taleb describes Antifragile systems as systems that become stronger when subjected to stress. For systems to grow under stress, some amount of instability is required. Boundaries provide that right amount of instability. We tend to make systems more and more stable and that backfires.
To put it interestingly, we must design a system like a Swiss government — Modular, bottom-up, with some amount of instability. Here is how Nassim Nicholas Taleb describes the swiss government.
Boundaries Allows You to Make Small Errors
“Things break on a small scale all the time, in order to avoid large-scale generalized catastrophes.”
— Nassim Nicholas Taleb, Antifragile
Nature loves small errors. They are the basis of evolution. Errors help the system evolve without causing the errors to affect the other boundaries. A part of the system might not work for some time, but the system as a whole will keep functioning. When the error is identified and fixed, the system becomes stronger than before.
Boundaries give room for trial and error. It is through some amount of chaos, strong systems are born.
Small Means Focused
I must confess that even after developing software for so many years, when I look at a system as one big thing, I get intimidated. Boundaries allow you to focus on parts without worrying about the whole, hence improving quality of work.
Boundaries Ensures Emergent Design
“It is not the strongest of the species that survive, but rather, that which is most adaptable to change.” — A quote often attributed to Darwin’s Ideas.
I cannot stress this enough — Architecting is a continuous activity. Software design is something that develops over time. Nature also draws boundaries in systems to help them evolve as a whole. Boundaries help components take separate evolutionary trajectories without stepping on each other and at the same time efficiently collaborate with each other.
The Creative Destruction of the Monoliths
“You must have chaos within you to give birth to a dancing star.”
― Friedrich Nietzsche
What is the point of building systems that don’t evolve? Systems need to be broken into smaller parts for them to become evolvable.
This is how you should design your Monoliths.
Create Boundaries Within a Monolith
This is the first and most important and the most creative step of decomposing a monolith. This is a bit of an artistic process and there are several ways of doing this, but I recommend doing it the DDD way. Eric Evans’ book “Domain-Driven Design: Tackling Complexity in the Heart of Software” tells us how it is done using concepts like Sub-domains, Bounded Contexts and Ubiquitous Language.
Chris Richardson has a few other interesting ways of decomposing Microservices by Business Capabilities. We can also apply it to create boundaries within Monoliths.
Use Packages to Separate Boundaries
Each boundary identified can be housed in a package of its own.
In the article “PresentationDomainDataLayering”, Martin Fowler talks about how the layers SHOULD NOT be used as top level modules, and only the boundaries or bounded contexts SHOULD be top level modules.
Here is the wrong way of creating Boundaries: Layers used as top level modules.
This is how the boundaries should be created: Boundaries or bounded contexts as top level modules.
In the diagram above, I am showing the structure inside boundaries modeled on a 3-Layered architecture. A Domain Driven Design based Hexagonal Architecture can also be used. In a DDD based Hexagonal Architecture, the Business Logic is at the center and outside shell is made of the technology layer that interacts with the world.
To keep the diagram simple, data models are not shown, but they also need to be isolated within their boundaries.
Continuously Verify the Fitness Of The Boundaries
It is fairly easy to create boundaries, but it is difficult to maintain them. This is where the violation of boundaries happens. The developers take shortcuts and try to access the classes of other boundaries, diluting the separation of concerns.
This can be prevented by using scope modifiers, but they have limited capabilities of avoiding referencing. They can be changed by the developers when they feel that they need it. For that reason, the Packages also don’t provide a serious level of separation. Entanglement risks are an ever present danger.
Wikipedia defines Fitness functions as “A fitness function is a particular type of objective function that is used to summarize, as a single figure of merit, how close a given design solution is to achieving the set aims.” Simply put, fitness functions are the measure of how close a solution is to its goal after each evolutionary mutation.
In the book “Building Evolutionary Architectures” Dr. Rebecca Parsons et al., say that fitness functions also applies to Software Architectures, “In software, fitness functions check that developers preserve important architectural characteristics. […] The fitness function protects the various architectural characteristics required for the system.”
It’s important that Architects define automated fitness function to protect the purity of the boundaries and the architecture. It pays to identify the fitness functions early.
Use Libraries like ArchUnit and Jdepend to detect architectural violations by developers.
Create Weak Links Between Boundaries
A system is never the sum of its parts. It is the product of the interactions of its parts.
— Dr. Russel Ackoff
Events are the most underutilized concepts in 3-Layered Architecture. Events that pass messages along with it are the most powerful way in which the boundaries can be loosely coupled.
For synchronous calls, one boundary can depend on the other layer using a well defined API exposed through a service layer. But, make sure you avoid circular dependencies.
Spring Application Events are a clean way of sending messages outside boundaries.
Communication between boundaries must be kept to a minimum.
Partition Tables in the Same Database
We can partition tables within the same database. If the isolation of the Model boundaries within the package are ensured using the fitness functions created by ArchUnit and Jdepend, the isolation of tables for modules is guaranteed.
If you are using an ORM, avoid object references in models from one package to another.
Join Query != Reporting
SQL join queries are not an optimal way of creating reports as they get slower with time. Reports need their own materialized views.
Data can be pushed to a reporting boundary through events that can create materialized views for reports.
The Next Step — Separate the Packages Into Jars
Once you have achieved some stability around boundaries, you can separate your packages into separate Jars, contained in a parent Jar.
This can be an intermediate step between your application’s evolution to Microservices.
This is where your Microservices Journey Should Begin
“If you can’t build a well-structured monolith, what makes you think Microservices is the answer?”
– Simon Brown
You will realize that till a certain point there will be no need for you to separate the boundaries into Microservices, but you will still reap the benefits of the Microservices. Evolving to Microservices will just be a step away without the need for re-engineering the project.
The Galls Law states that, “A complex system that works is invariably found to have evolved from a simple system that worked. The inverse proposition also appears to be true: A complex system designed from scratch never works and cannot be made to work. You have to start over, beginning with a working simple system.”
Starting a Microservice without first building a modular monolith will be like skipping a stage in the process of evolution. Take note — the reality will not be kind to you if you do that.
Conclusion
Starting a Monolith without proper planning and design is like falling for a Faustian Bargain. For initial progress, long-term evolvability should not be sacrificed.
Microservices are something you evolve into by the process of trial and errors. It’s cheaper to indulge in trials and errors when the boundaries are closer.
