Death to Monolith, All hail the Modulith

I’m old school. I started programming in C/C++, and programmed monoliths before I even knew what they were called. Back then we programmed monoliths without frameworks, which would have made it a lot easier. We were just programmers, period, and while we started with nice clean and structured code, after a while all of these monoliths became monstrosities which no one really wanted to take credit for. Ring a bell?

I’m sure you can imagine how excited I was when we heralded in the era of micro-services. This would be the solution to all of our problems, the answer to all of our prayers.

It was the best of times, it was the worst of times, it was the age of wisdom, it was the age of foolishness, it was the epoch of belief, it was the epoch of incredulity, it was the season of light, it was the season of darkness, it was the spring of hope, it was the winter of despair.

Charles Dickens, A Tale of Two Cities

A lot of promises were made:

1. No more stepping on each other’s code with endless merges
2. Choose any Language/Framework for any service
3. Pick & Choose different Databases
4. Full project life cycle ownership
5. No more scalability issues
6. …

The list goes on.

I have written many micro-services in my life, I can’t even count them. I even have a course I teach about it, and while I do love the micro-service architecture (MSA), there is always a sense of “doom” before setting out on the journey. Let me explain what I mean.

The Promise not kept

Whenever you start writing micro-services, you have to invest a lot in the infrastructure. Deployment, building the services, communication, databases, environment, scaling issues… you get the idea. Before you even begin touching the actual business logic you are heavily invested 2 months into the project.

After another month of programming, you start realizing you are missing a vital part of your system. Something that connects, controls and orchestrates. Basically, you realize you need an orchestrator. And that is after you tried the choreography approach and decided it’s way too complex and unmaintainable. And let’s not forget the SAGA pattern to compensate for the lack of 2PC. In short, a lot of “fun”!

You shouldn’t start a new project with micro-services, even if you’re sure your application will be big enough to make it worthwhile. — Martin Fowler, Monolith First

On the other hand, we did start with micro-services to escape the big bad spaghetti monolith, which wreaked havoc on our development velocity. So what do we do? Well, there is another option. The Modular Monolith, or in one word: The MODULITH.

A New Hope

This architectural pattern may just be the answer and the right balance between the monolith and micro-services. In this article, I wish to tell you why I think this is so.

Start with Business Logic, Not Infrastructure

When setting out writing a Modulith, you are focused on the business logic of the service, much less on the infrastructure. There is no HTTP or queue communication pattern to consider, neither is there a “what if the service is down” consideration. You are free to express your algorithm in one continuous stroke of a keyboard. The infrastructural demands and needs are much more sidelined and do not take up the main focus of the project.

On the other hand, what you MUST consider, is the separation of modules in a VERY STRUCT WAY. You have to define the boundaries of modules and stick to them like a flea to hair, like a mouse to cheese, like… hopefully you get the idea by now.

I suggest draconian rules to your build/testing process to ensure this behavior such as:

• ArchUnit for JVM language (https://www.archunit.org/)
• PyTestArch for Python (https://pypi.org/project/pytestarch/)
• TS-Arch for TypeScript/Javascript (https://github.com/ts-arch/ts-arch)

Easy extraction of modules into separate services

The beauty of this Architectural Pattern, the modulith, is its dynamic property. This is actually the selling point for the modulith rather than a classic monolith. If you adhere to the basic rule of separation of concern (SoC) you will be able to extract any “concern” or in our terminology: module, at any given point in time and make the system more scalable.

In a regular system, you have the more “managerial” API, user management, reporting, and other appended services, which receive a lot less traffic than the main API, the core of the system. In a monolithic architecture, this whole system would be one service, duplicated for the sake of the core functionality. In MSA, even the smaller, less used APIs would be self-managed/deployed creating many more services to monitor. Using the modulith as a strategy, the core functionality would be extracted as a separate service, while all other services would stay bundled together as one.

Considering the many services I have developed and overseen in my professional lifetime, I can safely say, that not all services are created equal. Some services do tend to be blasted by request and need to be more scalable, while others do not have a necessity to be a service of their own. The only reason some services exist as a separate service is by their belonging to a MSA.

This, in my opinion, is a “managerial” excuse. The reason for a service to be a micro-service of its own is by its own requirements and the necessity of the system, not because of a blind following of an architecture built for a subset of the whole system. As with many things, the truth lies much more in the middle, than at the extremes.

In a way, the modulith is much more “tailor-made” to fit the needs of the system.

Added Benefits in the Development Process

The benefit of working in a modulith, is the explicit API. This is something that has been lost in MSA, and many solutions try to make it easier, such as OpenAPI/Swagger, but there is nothing as easy as an IDE telling you what you can or must do. Not even AI. No fiddling around with JSONs and ORMs as good as they may be. It’s just… well, it’s just there! And when running or debugging (god forbid), you run your service and everything is there. No external calls to other services, no ORM/serialization code causing havoc. Straightforward!

I know. You all believe that developing MSA is much easier than a monolith. You work on your own service, nobody is bothering you, your framework, your language… No. I would argue: It depends. Mostly on the culture of the organization you work for. Let me give you some examples of how developing a MSA can be disastrous.

If your service depends heavily on other services and in order to run your service, you need to run other services as well. Not all companies have a great setup, where you can connect your service to those running in the cloud (and imaging setting something like this up!!!). Even worse you cannot run your service locally, but in order to test your stuff you have to deploy, which takes a couple of minutes each time you change a ‘;’. And let’s not even talk about not being able to debug. Or for instance, you are not the sole owner of the micro-service? In this case, you will enjoy the natural wonders of merging code just as in any other monolith.

The fact is, “stepping on one’s code” is a cultural issue, which can be mostly avoided in any architecture or code base.

Why solve a non-existent problem

This topic I want to explain using an analogy. Did you ever encounter someone explaining a new architecture, explaining where each paret is? It goes like this: “These are the micro-services, this is the database, and this is the cache layer…” And I always ask the same question: “Cache Layer? Why? Do you have a problem with latency?” The fact of the matter is that people like to solve problems before they even exist. Just because in the past architecture, caching was needed to solve a problem, does not mandate the use of a cache layer in this solution. How about solving the problem, not the thought of a problem?

For micro-services, it’s actually the same. People like to solve the various problems before they even exist. For instance, scalability. How many companies out there need to have all of their services meet a 10K RPS? Really? All of your services? And even if so, how about using a language/framework that can support this easily? I’ve been at too many clients that didn’t even consider any other approach than a micro-service one.

When asking why they are not considering any other approach, I usually get one of the following answers:

1. This is our culture
2. We want to be prepared for when this happens.

Culture? Have you ever met a company that stated that a hammer is their culture? Because if all you have in your toolset is a hammer, then of course every problem you will encounter will look like a nail. No arguing with that. But the other answer, about preparedness, this is exactly what I am talking about. Start with the business logic. Solve the problem, while keeping preparedness for when you need to solve the scalability problem, or any other for that matter.

Conclusion

It seems so long ago when we have programmed a monolith, that we have forgotten that it too, has benefits. For some reason, we remember much more the problematic and painful parts of the monolith than we do the good parts. But guess what, I’ve come to confess: Programming MSA is not all roses either. In fact, it’s quite hard, and other prominent and wise people even agree with me.

The Modular monolith is trying to bridge the gap between both worlds, and although it is not a hype (yet) to use the modulith architectural pattern, I suggest you give it a try. You might even like it. ;-)