Microservice Release Management
Mono Release or Micro Release? CI-CD for Microservices
Mono-Release Train
We have been following the strategy of treating the docker image tags of all microservices at an event as a single release. This release can be installed into a cluster by some tool, though effectively only the changed docker images get updated.
The event referred above is tied to the CI/CD flow.
A merge to master (pre-merge) could trigger a CI/CD event that could trigger a Docker image build and version upgrade of a particular MS, and a test release created out of the updated MS and rest of the unaffected MS. This test release could be applied and tested in staging.
Similarly, the event could be a post-merge event, in which case the release could be a candidate for deployment to production after tests. We can call this pattern mono-releases. This is what we have been following for about two years and I can tell that this gives a lot more clarity and sanity for all, especially program management who can think in conventional releases.
I am explicitly not mentioning here about versioning the interface or API exposed by each MS. For MS to work properly you need to ensure that interfaces are backwards compatible, and you should rely on strongly typed and versioned interfaces like GRPC instead of interfaces and types by conventions which can be easily broken. REST was never meant as an interface contract for a complex service.
There is a problem with this versioning of a set of MS. What if you have thousands of microservices. In this case, the rate of change will be so high that it will be difficult for a CI/CD system to keep up with the mono-release pattern. By the time you tag and send it to CD machine for a test, there will be a hundred other MS changed in master, and you need to batch a snapshot from the master, where the minimum batch time will be equal to your CI.CD time. You have to do this serially then as you are snapshotting a release in time and testing it and promoting it.
Micro-Release Train
What about each MS going through its own CI/CD cycle, that is each or a common CD environment is updated by the latest published docker image version of MS in production registry and your MS from dev registry tests against this and if it passes then it is published to say production registry. This will be like a micro-release pattern. Instead of each MS, you could group similar MS together too...
Do add your thoughts or comments if you have experience in this latter micro-release and the challenges that need to be overcome here. Running test cases in parallel would mean that test cases should be in such a shape. Also, it is clear that we need an efficient cluster manager like Kubernetes to update the pods/ docker images in a dynamic way as automation test cases that are running continuously would be dependent on these microservices. Clearly this train is challenging but more efficient and maybe finally the only way when there are too many microservices and too many changes happening.
