How we planned for monolith decomposition at QuintoAndar
An experience report on how we created a cohesive plan for decomposing the company’s monolithic application
In the early days, tech companies usually tend to develop digital products rooted in monolithic architecture principles. Such an approach provides lots of benefits when resources are limited and features need to get done quickly.
Monolithic applications are simple to develop, test, and deploy. A single codebase allows developers to easily apply changes, retrieve all necessary data for any business rule, define end-to-end tests, and ship solo artifacts to multiple servers (simple to scale).
However, “there is no free lunch” [at all times]❗️🤔
At some point, fast-growing companies will soon face important limitations around the monolithic architecture. As the business expands, many additional capabilities will be added to the application, making the codebase larger, complex, and very sensitive to subsequent changes. This process is what Chris Richardson defines in his book as “the slow march toward monolith hell”. 🔥
At QuintoAndar, we also have a monolith application on which the first business features were developed and are still working. In the last few years, following the company’s growth, we’ve been able to tackle down multiple complexities around the monolith by migrating it to the microservice architecture (microservices).
Promoting that architecture has helped us to decompose the monolith into a set of focused services, such as signatures, credit analysis, and owner fees management systems. This modularity enabled teams to be more autonomous and empowered to create new isolated and resilient features. Microservices proved themselves to be a suitable strategy for us to keep up with the company’s growth.
Yet, our monolith is a huge system with very particular implementations. It hosts many of QuintoAndar business contexts such as users, houses, and rental management. Therefore, a complete decomposition is expected to be arduous and resource-consuming.
After the first organic experiences, we perceived that each service decomposition required distinct amounts of planning, time, and human resources. In that context, we urged the next decompositions should pursue the best business gains at reasonable costs. Moreover, organic decompositions (with no organized plan) could put our microservice architecture at risk by creating a distributed monolith instead. A distributed monolith may leverage tools like Kubernetes and distributed architecture, but because services were poorly designed, they suffer from inefficient reuse, low cohesion, high coupling, and weak service contracts.
To achieve a cohesive decomposition plan, we decided to work on the following set of phased actions:
- Identify all bounded contexts residing in the Monolith
- Understand current AS-IS processes for each bounded context
- Establish expected TO-BE designs for each bounded context
- Combine knowledge and compose the Decomposition Big Picture
- Define decomposition order based on an ROI curve
- Present a semantic plan to stakeholders
1. Identify all bounded contexts residing in the Monolith
Our monolith application became too complex for a single person to comprehend. Even a long-tenured engineer wouldn’t be able to state all of its business capabilities. So, implementation details tend to be shared across multiple teams and service owners.
To capture that knowledge, we leveraged from Domain-Driven Design (DDD) techniques to construct a high-level view of the subdomains and bounded contexts residing in the monolith. Such structure helped us in two forms:
- Visualize what is happening in the software
- Reveal the Ubiquitous Language used to express business capabilities.
The next image illustrates an example of a monolith’s core capability map that can be created through a series of event storming meets. Meets should be conducted with monolith experts from different teams. In the meetings, we wrote in post-its all nouns that represented a particular domain model, user story, or event in the monolith software. In the end, we grouped them to form bounded contexts.
Revealing such a structure made it a lot easier for us to navigate over the monolith components and investigate software details with a guiding purpose.
2. Understand actual AS-IS processes for each bounded context
Once we got an overall idea of the monolith domains, we dived into software implementation to better understand the actual AS-IS processes for each bounded context. Such understanding helped us to get how business requirements were being expressed in terms of software.
Software is the main source of truth for getting business processes right! So, code investigation was the key action in this phase.
Nevertheless, monoliths are known to have quite huge codebases. To set an accomplishable investigation, we narrowed it down to focus on the most important components:
- Models: software that represents the essential aspects of a domain.
- Services: software that expresses the business operations of a domain.
- Events: software that triggers when something happened in a domain.
- Related or Dependent Domains: domains that rely on domain data.
Documenting those components was the key deliverable, to share knowledge with other monolith experts in the next phases. The following image illustrates a kind of diagram we used to document software components per bounded context.
Having a final document structure was not a concern at first. So, we focused on quickly writing discoveries on any possible approach and then making progressive enhancements throughout the investigation.
Additionally, we used the codemr (codemr.co.uk) analysis tool for measuring and visualizing software coupling. Such information revealed to us the most coupled classes and domain dependencies happening in the monolith.
3. Establish expected TO-BE designs for each bounded context
Once we got enough understanding of AS-IS processes, we were better prepared to establish how contexts should be extracted from the monolith and migrated to a set of independent and distributed microservices.
In this phase, some questions guided us through the process of defining the TO-BE designs for each bounded context:
- Is the bounded context separably?
- If yes, how should it be implemented (TO-BE design)?
- If no, what are the constraints?
- What are potential issues for decomposing that bounded context?
TO-BE designs were meant to be solid engineering foundations on how the AS-IS process should be implemented in the new microservice architecture. In that sense, it’s quite important to promote broad design discussions with as many monolith experts as possible. To better guide discussions, we firstly designed initial sketches for each context. They served as starting points and were based on knowledge acquired from previous phases. The next image illustrates an example of how we documented part of a TO-BE design proposal. Along with that, we also tried to define an idea of which microservices would be necessary to implement business capabilities.
Additionally, we put in mind that TO-BE designs should be focused on promoting scalability, resilience, and modularity of the bounded contexts and new respective microservices.
4. Combine knowledge and compose the Decomposition Big Picture
After defining TO-BE designs for each bounded context, we consolidated all knowledge by composing a Decomposition Big Picture which highlighted main components and domain dependencies.
The Big Picture helped us to visualize how the monolith decomposition may look like at the ending of the process. Additionally, visualizing dependencies gave us the sense of which contexts or domains could be decomposed in parallel or not. The next image illustrates how we combined the domains in a bounded context to visualize the Big Picture of a core business foundation.
5. Define decomposition order based on an ROI curve
Another important assessment of our plan was to define the return over investment (ROI) for decomposing the monolith. For that perspective, we needed to plot each bounded context on a Cost vs Benefits graph to describe the ROI curve.
For the cost axis, we considered the following parameters:
- Code coupling: a measurement of the number of classes that a class is coupled to. Monoliths usually have high coupling between entities which requires more efforts to decompose and can represent greater refactors. Such metric was extracted from the CodeMR analysis tool.
- Extraction complexity: a measurement of a bounded context technical complexity, considering its codebase size and how it is used by others. High technical complexities require greater decomposition efforts. Such metric was calculated comparatively across bounded contexts by applying weighted values for each one.
- Business complexity: a measurement of the number of business rules that exist in a bounded context. Many business rules require greater efforts to reveal each context's nuances. This metric was calculated comparatively across bounded contexts by applying weighted values for each one.
- Domain dependency: a measurement of how much a bounded context depends on others for delivering its greatest business value. The more a context depends on others to meet its purpose, the more complex its decomposition may be. This metric was calculated comparatively across bounded contexts by applying weighted values for each one.
For the benefits axis, we considered the following parameters:
- Business flexibility: a measurement of how a decomposition will improve business flexibility of a bounded context. When extracting a highly coupled context, we expect that will be easier to evolve its business capabilities. This metric was calculated comparatively across bounded contexts by applying weighted values for each one.
- Reduce load on monolith database: a measurement of how much load (queries, insert, updates) the features of a bounded context weighed on the monolith database. When extracting a context, we also wish to reduce the amount of DML operations in the monolith, reducing database CPU and memory usages. This metric was calculated comparatively across bounded contexts by gathering inputs from database load metrics-related tables.
- Business fault-tolerance: a measurement of how fault-tolerant a bounded context is. When a context is non-fault-tolerant, it means that an eventual monolith outage will have a substantial impact on the context’s functional requirements. This metric was calculated comparatively across bounded contexts by applying weighted values for each one.
By measuring those parameters, we were able to create an ROI graph to comprehend the business values for each bounded context. The next image illustrates how we proposed to visualize the ROI curve. Each point combines the cost and benefit values of a bounded context decomposition.
6. Present a semantic plan to stakeholders
The final step is to communicate the decomposition plan to align strategies, priorities, and expectations across multiple stakeholders. In that way, we created a semantic overview to explain motivations behind the monolith decomposition, efforts to get it done, and business values for the company's growth.
To present the semantic view of our plan, we gathered the company’s main business foundations and highlighted the bounded contexts that are still residing in the monolith. That provided an objective view of the important features that still depend on the monolith application and should be decomposed into microservices. The next image illustrates the semantic decomposition order of domains and bounded contexts residing in a monolith core business.
📚 Learned Lessons
In the beginning, we didn’t have a definitive perspective of all steps to cover for the plan. So, our approach was to get started by combining knowledge from multiple literature references and quickly iterate over the first discoveries as soon as possible. The first phases were confusing, so we had to change directions fastly which made us find other proper approaches throughout the process.
Moreover, we learned how to investigate codebase much faster. It is not possible to get all coding details, so it was essential for us to focus on the main components defined in phase 2. With that in mind, we were able to quickly iterate over the codebase and extract just enough valuable information for each context.
Finally, we learned how important is to well communicate the decomposition plan across multiple stakeholders. We worked on delivering a semantic explanation of the plan by removing unnecessary jargon and simplifying goals alongside business gains in clear language. Simplifying complexity pays off!
💭 Final Thoughts
After coming up with a cohesive decomposition plan, we feel more confident to address QuintoAndar business scalability by assertively breaking the monolith down into independent, flexible, and distributed microservices.
Additionally, in the long term, we now have a more consistent view of the monolith’s future and how its decomposition is going to enhance business capabilities while the company continues to grow.
For the next phases, we’ve chosen to decompose a cross bounded context based on the order and ROI values generated by the plan. In parallel, we also started to define microservice blueprints that will be referenced for creating new microservices and maintaining existing ones. By including these blueprints in our decomposition process, we expect to consolidate our microservice architecture and ensure that we're running away from the distributed monolith.
That’s all for now folks! Hope you’ve enjoyed learning from this experience and had interesting insights on your next monolith challenges. Bye. 👋
