Preparation to migrate a domain from a monolith into its dedicated service
When starting to build a new product as an MVP (minimum viable product), it’s hard to foresee how its development will evolve and what features will become relevant after releasing it to the public. Once the product is out for consumption, users or clients will help guide what functionality is important and how the product should evolve.
As a small group of developers initiating the project, while the product is in its validation stage, it is a completely valid approach to start simple and allocate all the logic under a single service to reduce costs, simplify operations and move at a faster pace to deliver a working product quickly, sometimes even hastily.
The fact that the product should be released in a short period of time, doesn’t imply developers should not care about establishing a good and extensible foundation for the application growth. Otherwise at a certain point in the future, they might have to pay a higher price back when the product needs to be extended with new features and functionality needs to be extracted into new services.
Over time, the more functionality is added to the service, the more it will grow in complexity, business logic and different kinds of data to manage. If that preliminary foundation is not solid, unless measures are taken at the right time, the service could end up convoluted, with coupling in the code base producing an entanglement between several domains.
Having more than one domain within one service could complicate the application management. As the number of developers that work on the product grows, new developer teams will be spawned, with more than one team working simultaneously on the service’s development.
Ideally, the system’s architectural design should reflect the organizational structure, as stated in Conway’s law. This will prevent unnecessary communication overhead between teams, reduce cross-domain dependencies in the code base and lessen overlapping in deployment processes and other factors, that can ultimately affect the development teams’ autonomy and velocity in delivering changes that improve the product.
Once this situation is given, the aforementioned issues could be alleviated by extracting certain functionality from the monolithic service in which all these domains live to its dedicated domain-specific service. This newly born service should have a single, clear responsibility and ideally one team should be responsible for its development and maintenance.
Until the new service’s birth and its production usage, there is an often tedious and complicated process that requires the preparation of a clear vision and strategy before its development can even start.
- The vision will help to think ahead about the evolution of the architecture and also reduce potential pitfalls.
- On the other hand, the strategy will aid to minimize failures along the way, smoothing the execution process and avoiding the unexpected emergence of unplanned work. With that said, there will always be those unforeseen tasks that will need to be addressed throughout the journey.
The following 10 steps should assist you in preparing the migration of a domain to its dedicated service.
1. Define the components and domains contained within the monolith
As a first step, ensure the monolith’s components are well understood by all developers who work on it. Even if developers have been working on a service for a long time, this doesn’t mean that everyone is able to zoom out to fully understand other parts than the ones they are used to working on. And even if someone did, the understanding might diverge between different people until there’s a conversation to reach consensus.
Therefore, as a good point of departure, identify and document the components that live in the monolithic service in order to obtain a shared understanding with other engineers and group them in domains.
- A component is a building block of the application, with a clear role and responsibility and generally, constituted by multiple classes.
- A domain could be interpreted as a context in the real world in which the software solves a problem.
A good way to start having an open discussion, in the path to find an agreement on the shared vision within the team, could be hosting an Event Storming workshop to convey ideas and different points of view in an interactive manner. The domain definition should be a collaborative process, in which even non-technical people who are domain experts can participate.
Another great tool, in addition to an interactive workshop, is a written proposal. Written proposals are a great way to collectively agree on solutions in an asynchronous manner, giving people more time to quietly think at their own pace without the social pressure to construct justifications for their reasonings. Furthermore, proposals can also serve as documentation of the outcome as well as for the discussion process.
Component interdependencies
At this stage, it’s also worth investing time on analysing the interdependencies that exist between components and domains. In case of a domain extraction, these dependencies will eventually be translated into either data duplication across services or communication between services.
2. Identify the domain to be extracted
Knowing all domains that comprise the monolith and having a clear overview about how they interconnect, it’s a good foundation for making a better-informed decision on which domain and its subjacent components will constitute the new service.
A domain doesn’t necessarily need to be reduced to a single component. Don’t fall into the trap of spawning a micro-service per component, just because it seems to be a tendency in popular companies in the industry. Instead, give it enough thought before concluding, with strong reasons, that there are benefits for going with this solution. Otherwise, one side effect is that the system’s architectural and operational complexity could increase in an undesirable manner for the team that has to maintain it.
In this decomposition process, it would be advisable to start with macro-services, referring to service-based architecture, where more than one component can be aggregated into a single domain for reasons such as logic sharing, semantic cohesion, component entanglement, etc.
After the service is created, it will be clearer whether there is an actual need to further decompose it into subdomains or whether the architectural state is good enough to remain as it is.
Architecting for change
If there’s a need to reduce complexity in a monolithic service by extracting a domain, an important aspect to account for is the likelihood that the domain’s code will be changed.
Ask yourself: “Will the code that belongs to the domain be subjected to frequent changes or will it remain untouched, simply requiring some maintenance from time to time?”
Once the service is running, features and improvements relating to that domain should be delivered to end users at a higher velocity, reducing the time to market. This is often a good reason for working in such a migration, as it promotes strategic advantages for the business.
3. Outline the business value
Re-architecting services and extracting a domain from a monolith into its own service is a tedious and complex task. This work might require weeks and sometimes even months, and therefore, this labour should be backed by justified business value that the company will benefit from.
Start focusing on the goals, then on the process.
The decomposition of a monolith into several services may aid to improve certain system quality attributes such as maintenance, velocity, autonomy and many others. But are those aspects relevant for the business at the moment in time when the work is being proposed?
There may be other, more appropriate topics to tackle that could bring either more economic value or strategic advantage over business competitors at a given time. Besides, system quality attributes are technical terms that might seem abstract and not easily understood by non-technical stakeholders.
Consequently, it’s worth investing effort in finding the right words to describe the impact that the labour will have on the business. If such a statement is backed with numbers, even better.
As an example, imagine a hypothetical case in which a company has the need to double its user base and currently is at its capacity for dealing with its existing user base load. What would stakeholders think if the following statement were presented?
It is estimated that this migration will take around 2 months. It will help us to support twice the number of users we do now, as well as accept 10,000 orders more per minute than the 15,000 we are currently supporting. This could potentially translate into $400,000 in increased revenue.
A sentence of this kind will help depict the benefits to everyone. However, once non-technical stakeholders see the impact on the business, it will help convince them about the importance of this task, making it easier to buy them into the idea of working with the migration.
It’s definitely not easy to come up with such a statement and the numbers for this kind of work. Nonetheless, the takeaway is to simply think about translating them into a language that everyone understands and that clearly illustrates the business value.
4. Envision the service evolution
Thinking about the service evolution could be a speculative process, since it’s not possible to anticipate future business requirements with a high degree of certainty, although it will help establish a more solid foundation for the new service, along with measures to prevent eventual pitfalls.
Ask yourself questions such as:
- Will the service need to scale to thousands of requests per second at some point?
- Will the service need to be extensible to support new features or connectivity with other services?
The purpose of these questions is to identify risks that could affect the architecture’s evolution. It is recommended to document and analyse potential problems with the purpose of finding mitigation strategies while preventing dead ends in the future when problems could arise.
5. Assess trade-offs
It’s impossible to come up with an architectural solution that complies in the best manner with all non-functional requirements (a.k.a. system quality attributes), such as scalability, cost-efficiency, extensibility, etc. within a finite amount of time.
Instead of trying to comply with every single system quality attribute, be pragmatic and understand what are the most important ones. Then, propose a solution that complies with them as much as is feasible. It’s not necessary to invest time in making an API respond under 50ms for all its endpoints, if performance is not such a critical factor.
Software architectures are never perfect — they were built by assessing trade-offs that might have been valid compromises in the past and whose aim was to promote the focus on what was important for the business at the time when those decisions were made.
6. Specify technologies for the new service
When creating a new service, it’s a good moment to re-consider technologies. New technologies should be introduced if they bring a tangible benefit and not just because they are trendy and well known companies are using them.
Introducing a new technology that most developers are unfamiliar with will require a period of learning, practicing and mastering, slowing down the development’s velocity at the beginning. Furthermore, the more technologies are introduced in a company, the harder it gets for existing staff and new hires to maintain them.
Make sure the team that will build and maintain the new service will feel comfortable with the technical decisions made in advance.
Programming language
Engineers might decide it is time to try that new trendy language without having thought about the implications that this decision will entail. There’s nothing bad about being innovative and wanting to introduce a new and more modern language, although there are trade-offs to be aware of.
In the scenario in which big portions of code from the monolith can be reused and extracted to the new service, it might be advisable to keep using the same programming language in order to save a substantial amount of effort.
Replacing the programming language will generally imply rewriting the code, unless a decision that allows a gradual transition is taken, such as transitioning from Java to Kotlin or from C to C++.
On the other hand, there might be strong reasons for ditching the current language and applying a different one at the expense of a potential code rewrite.
If developers that will work on the project are unfamiliar with the new language, this will most likely decelerate its delivery. Under these circumstances, it will be convenient to have at least one expert who can instruct others along the way and ensure the quality of the software delivered.
Carefully evaluate if the delay of rewriting everything into another language is going to bring benefits worth considering, versus the time it will delay the delivery of the project.
Frameworks and libraries
If you decide to stick to the same programming language, it might be worth re-evaluating other frameworks and libraries to obtain certain benefits that were lacking in the ones being used in the monolithic service.
Considering switching to a new programming language, if the company relies heavily on open-source code, ensure that the language is mature enough and has a strong community that builds good libraries and frameworks around it.
Patterns
This is a good moment to evaluate new patterns, since they influence the internal structure and design of a service. New patterns could bring advantages, such as making the project more maintainable and understandable, the code more extensible and reusable and other facets which will improve the overall development process.
If you decide to use the same programming language, the introduction of a new pattern should allow the reusability of code that was planned to migrate to the new service, although it might need slight modifications before it can be plugged into the new pattern interface.
Underlying hosting infrastructure
Changing the way a new service will be hosted, compared to how the monolithic service is, could bring certain advantages. Considering a few different hosting options, such as:
- PaaS (Platform as a Service) to reduce operational work and focus on application development to evolve the product at a faster pace.
- FaaS (Function as a Service) to have a cost-effective, scalable solution out-of-the-box and hassle-free server management.
- Self-managing infrastructure components, if there’s a need to have more control over them for particular business reasons.
- Migrating to a container orchestration solution, such as Kubernetes, in which all services can benefit from powerful DevOps tooling and be managed homogeneously.
- Moving from on-premises to a cloud hosting provider with the objective to leverage on-demand computing and don’t have the need to pay up-front for hardware resources.
Ultimately, this change might result in additional work that should neither be underestimated nor forgotten. This effort could include the following kinds of tasks:
- infrastructure resources monitoring and visualization,
- collection of application metrics,
- distinct alerting mechanism setup,
- different service management procedures,
- ability to improve deployment processes,
- software patching, or
- introduction of a new technology into the company.
Database
A domain is broadly constituted of entities and their associated data, but in certain cases it could be purely functional without requiring data storage. If the former applies, data will eventually need to be migrated to its dedicated domain service.
It might be favourable to reuse the same database technology, unless there is a solid argument not to. Using the same technology will reduce entropy in the data migration process, in which data will be transferred from the monolith database into the service-dedicated database.
Ideally the data migration process should be conducted without incurring downtime and with the ability to roll back use from the new to the old database in case of certain malfunctions. There are tools and cloud services that help deal with the migration of existing data and the synchronization of new transactions without causing downtime.
7. Forecast costs
Systems should always be built in the most cost-efficient possible manner and consequently, cost-awareness should be an important constituent in the decision process. Even if the budget is unlimited or a process isn’t in place for approval of the cost allocation by the finance department, costs that the system re-architecture will entail should be forecast and made transparent to stakeholders.
Increasing the amount of services generally increases the overall costs. Nevertheless, nowadays more modern hosting and cost-effective measures exist, such as serverless architectures, which may ultimately lead to lower overall costs.
8. Define the strategy
Describing the steps that can be foreseen in chronological order will certainly improve the execution process and reduce surprises along the way. In order to define the steps concisely and with an acceptable level of detail, an allocation of resources will be necessary to prepare the execution plan. This effort includes analysing the code, drawing diagrams, writing proposals, discussing different approaches and many other activities that should be conducted collaboratively.
Some steps that are generally applicable to most cases:
- The domain’s extractable code might be entangled with other domains. As groundwork, ensure that the domain’s code is decoupled before starting the migration.
- Evaluate how reusable the code is to reach a decision on either migrating it, migrating parts of it or a full code rewrite.
- Document in detail the new service’s API before starting its implementation.
- Define the order in which the logic can be migrated if there’s a possibility of tackling this gradually.
- Investigate the complexity of toggling between functionality in the new service and in the monolith, aiming for a gradual rollout plan rather than an “all at once” approach.
- Prepare how the functionality of the new service can be tested independently and in conjunction with the monolith.
- Ideate a database migration strategy.
- Define how the new service will be managed, accounting for monitoring, alerting, deployment strategies and all other related DevOps processes.
There are definitely more steps or tasks to consider, and every migration will demand its own particular ones. Be aware that even after having defined all the steps in as much detail as possible at an early stage, unforeseen issues and additional labour will surely arise as the migration goes on.
9. Estimate the effort and plan the work
It’s not easy to estimate the effort that will take a migration from a monolithic service to a new dedicated service, considering the amount of steps and unsought obstacles that may be encountered down the road. Having prepared a thorough strategy in advance will undoubtedly serve as a good foundation for estimating the effort. The more precise the strategy is, the better the estimations will be.
As a general rule of thumb, when estimating effort, evaluate with a pessimistic rather than an optimistic inclination. We all know that tasks will usually take longer than one had anticipated.
Every team operates with its preferred estimation units to measure effort or complexity, using fibonacci sequences, T-shirt sizes, story points, etc. However, for the purpose of high-level task estimation, I personally find developer days a better measurement, given it is a widely understood unit.
After the task estimation is completed, the next step is planning, where tasks should be scheduled in the chronological order defined in the strategy. Important factors to ponder in this phase will be the parallelization of those tasks as well as the availability of those developers who will be allocated to the project.
It is important to account for unforeseen work by adding a buffer to the different migration phases that is proportional to their complexity.
A Gantt chart will be a good ally in visualizing the planned work, seeing what tasks can be parallelized and the amount of people that will work on it. The visualization will contribute to an understanding on how long a presumably homeric migration will be expected to take and when certain benefits can start having impact.
10. Document decisions
Architecture decisions are trade-offs for system quality attributes that aim for the best compromise solution that satisfies service needs after taking their business importance into account. These decisions should be documented and chronologically ordered to assist with the understanding of how the architecture of a system has evolved over time.
Decision records should be created for architecturally significant resolutions, such as those that affect the structure, non-functional characteristics, dependencies and service interfaces.
There are different templates which can be chosen to create decision records depending on how detailed the decision needs to be portrayed.
Regardless of the template selected and the level of detail to be logged, these records should encapsulate the following core concepts:
- the context in which the decision was made,
- the problem solved,
- the solution taken,
- and lastly, the consequences and downsides of that solution.
Conclusion
Before starting the extraction of a domain from a monolith to its dedicated service, it’s worth investing time in the preparatory work and being thorough enough. This will enable more informed decisions, thereby preventing eventual pitfalls that could turn out expensive for the company.
Stakeholders should understand and be aware of the business value this labour will bring, as well as the estimated amount of time and effort it will take.
All the steps should be conducted as a collaborative process in which decisions are discussed and agreed upon with relevant teammates who can contribute to a better outcome.
Good luck is the result of good planning, hence you should plan thoroughly ahead of time to ensure a smoother and more trouble-free migration process.
