Kubernetes Application Deployment/Rollout Strategy

Application Rollout Strategies — Kubernetes & Istio

As most modern software developers can attest, container orchestration systems such as Kubernetes have provided users with dramatically more flexibility for running cloud-native based applications as microservices on physical and virtual infrastructures. Sometimes deploying an application to all machines in the environment at once, it is better to deploy the application in batches. In production environments, reducing the downtime and risk while releasing newer versions becomes business critical. This is especially critical in production applications with live end-user traffic all the time like a website and related components which are critical to business, such as a e-commerce, banking or investment website.

These applications almost always have frequent deployments. For example, a mobile application or an consumer web application may undergo several changes within a month. Some are even deployed to production multiple times a day. The speed at which you constantly update your application and deploy it’s new features to the users plays a significant role in maintaining consistence.

Application deployment becomes more complicated the more the product grows. Deployment of a new application thus may mean deployment of new infrastructure code with it. A deployment strategy determines the deployment process, and is defined by the deployment configuration that a user provides while hosting the application on Kubernetes. Some of them are mentioned below:

Recreate

Although this is not a true continuous deployment strategy which involves a stipulated downtime. This is more like a old fashioned deployment/installation process where this approach comprise cleanliness and conceptual simplicity. At no time users don’t have to manage more than one application version in parallel.

Application-V1 is removed once Application-V2 is introduced and deployed. There is no need for Intelligent routing and traffic switching mechanisms as there exists only one version.

Kubernetes Application Rollout using Recreate

Definition

Defining Recreate Strategy in Kubernetes Manifest

Working Pattern

Recreate — Working Model

As seen below the traffic is totally directed to App-V2 once App-v2 is rolled out.

Traffic Shifting Visualization — Recreate Strategy

This strategy is mostly appropriate for non-critical systems where downtime which depends on both shutdown and boot duration of the application. is acceptable and comes at no significant cost.

Rolling (or) Ramped Deployment

Rolling deployments also known as incremental are release patterns where the application is gradually deployed to the machines one at a time or in batches. App-V2 is slowly rolled out replacing App-V1 instances one after the other until all the V2 instances are completely rolled out turning App-V1 turned off. Ramped deployments are great when the new versions are backward compatible — both API and data-wise.

Kubernetes Application Rollout — Rolling Update Strategy

The number of simultaneous deployment targets in a rolling deployment is referred to as the window size. A window size of one deploys to one target at a time, and that deployment must finish before another deployment starts. Kubernetes Deployment supports rolling-update parameters mentioned below to streamline the deployment time and rollout process:

  • Max surge: How many instances to add in addition of the current amount.
  • Max unavailable: Number of unavailable instances during the rolling update procedure.

Definition

Defining Rolling Updates — Kubernetes Manifest

Working Pattern

Rolling Update — Workflow

As seen below the traffic is gradually shifted to V2 until V2 is completely/fully rolled-out. At any point of time based on the max-unavailable value set there will not be any downtime as at least one instance will be serving the traffic.

Traffic Shifting — Rolling Update

Shadow

Shadowing is a deployment pattern where production traffic is asynchronously copied to a non-production service for testing. This is particularly useful to test production load on a new feature. A rollout of the application is triggered when stability and performance meet the requirements. App-V1 receives real-world traffic alongside App-V2 and doesn’t impact the response.

This requires Istio which enables mirroring of traffic through istio-ingress-gateway.

Shadow Strategy — Kubernetes Application Rollout

Definition

Defining Shadow Rules — Kubernetes Manifest

Working Pattern

As shown below all the traffic to ‘my-app-v1’ is mirrored to ‘my-app-v2’. There is a Zero production impact. Since traffic is duplicated, any bugs in services that are processing by ‘my-app-v2’ have no impact on production.

Istio — Workloads and Services
Istio Rule — Virtual Service
Traffic Monitoring using Istio-Kiali
Traffic Shifting — Shadow Strategy

A/B testing using Istio

A/B testing allows running multiple variants of functionality in parallel, so that through analytics of user behavior, the better variant can be determined. Rather than a deployment strategy, it is usually a technique for making business decisions based on statistics.

The difference between blue-green deployments and A/B testing is A/B testing is for measuring functionality in the app. Blue-green deployments are about releasing new software safely and rolling it back predictably.

A reverse proxy capable of managing Layer 7 traffic (Envoy alongside Istio) is necessary to support these capabilities. Incoming traffic is routed through the reverse proxy, which then routes traffic to different versions of a service based on the rules set.

A/B Testing — Applications on Kubernetes

Definition

Defining Destination Rules — Kubernetes Manifest

Working Pattern

As shown below a weightage is associated to each version using Istio-Virtualservice object which distributes the traffic to the versions based on the value. Users also can use headers to direct the traffic based on version information.

Workloads and Services on Istio with a Virtual Service
Traffic Flow
Traffic bifurcated based on Istio Rules
Traffic flow Visualization

Blue/Green Deployments

Blue-Green deployment approach involves managing two production environments, as identical as possible. At any time one of them, let’s say blue for the example, is live. As users prepare a new release of software where final stage of testing is carried out in the green environment. Once the software is working in the green environment, switch the router/traffic so that all incoming requests go to the green environment — the blue one is now idle.

This can be achieved using native Kubernetes Service construct or using a ingress-controller. Shown below is the base minimal way of deploying a blue-green application where App-V1 is identical to App-V2 and the traffic can be easily shifted to App-V2 using the ‘selector’ label in Kubernetes service.

Blue/Green Strategy — Applications on Kubernetes

Definition

Defining Blue/Green rules — Kubernetes Manifest

Working Pattern

Both App-V1 and App-V2 are configured as endpoints under a service. Kubernetes Service manifest contain a ‘selector’ parameter which streamlines the traffic flow based on the ‘version’ label.

'{"spec":{"selector":{"version":"v1.0.0"}}}' and '{"spec":{"selector":{"version":"v2.0.0"}}}'

As shown below once App-V2 is introduced all the traffic is shifted from App-V1 to App-v2.

Traffic Shift Visualization — Blue/Green Strategy

The fundamental idea is to have two easily switchable environments to switch between, there are plenty of ways to vary the details. With this approach users can avoid versioning issue, as the entire application state is changed in one go and on the other hand this approach is expensive as it requires double the resources.

Canary Deployments

Canary deployments involve rolling a new release incrementally to a small subset of servers side by side with its Stable version. Once the functionality and stability are tested, the changes can then be rolled out to the rest of the infrastructure.

This effectively exposes new versions to a small set of users, which acts as an early indicator for failures that might happen. Canaries are very handy for avoiding problematic deployments and angry users! If one canary deployment fails, the rest of your servers aren’t affected and you can simply ditch it and fix the root cause.

As shown below the traffic is gradually incremented to App-V2 finally removing App-V1.

Canary Strategy for Applications with Istio

Definition

As shown below Istio’s ‘canary-weight’ is used to gradually shift traffic form App-V1 to App-v2.

Defining Canary — Applications on Kubernetes
Weighted Traffic Flow with Istio
Traffic Control with Istio to Versions

Working Pattern

With Canary-Weight ‘10’ to ‘App-V2’:

Traffic Visualization With Weight Paramaeter — Canary-Weight ‘10’ to ‘App-V2’

With Canary-Weight ‘100’ to ‘App-V2’:

Traffic Visualization — Canary-Weight ‘100’ to ‘App-V2’

Although this approach involves slow rollout, this is convenient for error rate and performance monitoring.


Users expect applications to be available all the time and developers are expected to deploy new versions of them several times a day. Strategic deployment of applications significantly affect business benefits like: time to market is reduced, users get product value in less time, overall developer morale goes up etc. Traffic flow management capabilities and container lifecycle management are some of the key advantages of Kubernetes and Istio where users can easily implement traditional software deployment strategies in microservices ecosystem.