Some Notes Of Microservice Migration
Microservice architecture encourages decoupling a system into physically independent applications. They have their own database, running environment, and release cycle. And they can even be implemented by different technology stacks. A full business process is completed by messages exchanged between these applications.
Microservice has benefits and costs. The benefits usually outweigh the costs when a system is big enough and its modules have complex coupling. To avoid unnecessary cost of microservice architecture, our team started from tiered monolithic application, and we decided to migrate when we found unnecessary deployments because of bad coupling. In this post I will describe how we migrated our services using an example. The example is not related to our production code, but it demonstrates similar issues we faced during development.
Imagine there was an application help you setup transfers among multiple bank accounts. The app has integrated with many banks and provided an unified UI to help move money around. It looked like this:
And entity AccountTransfer mapped to MySQL table.
AccountTransfer held information required by our app and bank APIs, which was passed into BankTransferService to do transfers. BankTransferService had multiple implementations for banks, communicating with their APIs.
Also, both services access account_transfer table, so they were coupled by DB. This logic is not shown in above code, but we had it to track transfer state and run recurring tasks within the services.
Identify Release Requirement and Define Service Boundary
At the beginning, we released our new feature with external API updates altogether. However, this pattern started giving us trouble when the scale of integration grew. Hotfix became more often, deployment happened frequently before scheduled features are merged. And we spent more energy managing changes and risks introduced by deployment.
Seeing this fact, we decided to separate the app to reduce the coupling. BankTransferService has been an obvious boundary for decoupling, so we moved out this class and made it the gateway of the new microservice to reduce release scope and make releases more safe.
Decouple Shared Domain Entity Into Interfaces
To physically separate into services, the entity shared across service boundary, AccountTransfer, also needs decoupling. We created new DTO interfaces for different services. and changed depending codes to use them.
And created new service API.
Note that AccountService and BankTransferService still communicated using the same AccountTransfer instance, but they accessed data via different interface, taking the first step to physical separation.
Divide Application Into Microservices
After we removed code dependency*, we could run BankTransferService on another process. Since then AccountService communicated with it by RPC instead of simple invocation. At this stage, AccountService and BankTransferService had a copy of AccountTransfer respectively, but accessed different part of fields.
P.S: more accurately, BankTransferService only provided AccountService interface-type API, which could still be accessed by AccountService code. However, interface gives us flexibility to change underlying implementation without worrying its dependents. We just need ensure every update is additional, and use deprecated interface to implement new ones when possible.
Refactor DB Schema
The final step of decoupling was separating the DB schema and made the services run in different verticals. We separated AccountTransfer into TransferRequest and BankAccountTransfer like this:
After the separation, we migrated corresponding DB fields of BankAccountTransfer to new table back_account_transfer, and account_transfer table stores TransferRequest. Considering possible rollback after new release, we copied migrated fields into two places —back_account_transfer table and the old columns in account_transfer. Only had the system become stable we cut off the duplication and complete the decoupling process.
We are still far from ideal microservice world. We didn’t build our system as microservices from the ground up, we have no automatic end-to-end delivery, and we don’t force team to push change into mainline frequently. However, considering the complex system we already have, and the risks induced during re-architecture, we believe it’s reasonable to stay in the middle ground for a longer period and carefully improve our system and working process.