Zero Downtime — Data in Microservices architecture series (Part 2)
This post is the second part of Data in Microservices Architecture series. You can find the prequel here.
When you release a new version of your software to your customers, one thing you want is not to disrupt their activities. That is what zero downtime is all about in your software deployment pipeline.
Without zero downtime you would schedule the hours of the day when you think your users are less active in your software. With zero downtime, you will have the ability to deploy at any time of the day, multiple times and with small batches of change.
In my previous post, I mentioned that the main driver on why Ralali.com moving to microservices is to decreasing our lead time and reducing the changes in software releases. In the near future, I envisioned Ralali.com system, that empowering MSMEs, is a distributed system with a lot of moving parts where unavailability caused by a broken deployment of one of its parts is not allowed.
Deployment Architectures
The load balancer has become a default in Ralali.com deployment architecture. This is a good starting point for us to exercise two deployment strategies that could help us to achieve zero downtime.
Blue/Green Deployment
Fun fact: Since May 2019 Ralali.com moved into the ‘Blugreen’ building in West Jakarta.
When you choose blue/green deployment you will have two different releases, current and new version of your application running concurrently. Both releases are running on two identical environments with each being capable of handling 100% of your incoming traffic on its own. This is represented by the blue and green deployments where blue as the current version and green is the new version of your software.
In a common scenario, the load balancer is forwarding requests to the blue deployment. After you completed a deployment of the new version in the green deployment, you can configure the load balancer to forwarding requests to the green deployment and stop forwarding to the blue one. The change must be made on-the-fly to avoid requests lost.
When you found anything wrong with the green deployment, you can reconfigure the load balancer to forwarding requests back to the blue deployment without having to wait for it to be deployed again. This reduces the risk of your software deployment process.
One important aspect of the blue/green deployment strategy is your software releases must have forward and backward compatibility so both versions can coexist at the same time. This means two things to ensure. First, you code that exposed APIs must retain compatibility. To gain this, you must implement API versioning rigorously. Second, the data (state) must be readable and writable by both versions in a consistent state.
Canary Deployment
You are a coal miner from many years ago where you don’t have the luxury to use modern sensor safety equipment. A common problem that presents to you is toxic gases that build up in the coal mines and some of them are not have an odor. So how would you alert yourself when the dangerous gases check-in? You bring caged canaries into the mines. While the canaries singing cheerfully you can assure the air you breath is fine. If the canary died, then you know it is your time to get out fast.
The canary deployment borrows the above analogy. The most important things are gradual deployment and monitoring. By gradual deployment, you would forward a small percentage of the traffic to the new version of your software. When you see nothing goes wrong then you would gradually add more traffic. If you see something is wrong, you roll back 100% of the traffic to the current version of your software.
To be able to route a certain percentage of the traffic to your software and to roll back when something goes wrong, you need a smart router or implement a feature-toggle framework. The feature-toggle framework allows you to choose which part of your software will be enabled to a certain percentage or group of users.
—
If you enjoy this post and want to be part of the Ralali.com mission to empower businesses and people in Indonesia, why don’t you visit our Career Page and see where you can fit.
