Continuous Deployment with Azure Kubernetes Service, Azure Container Registry & Jenkins

Introduction

Containerization has taken the application development world by storm. Kubernetes has become the standard way of deploying new containerized distributed applications used by the largest enterprises in a wide range of industries for mission-critical tasks, it has become one of the biggest open-source success stories.

Although Google Cloud has been providing Kubernetes as a service since November 2014 (Note it started with a beta project), Microsoft with AKS (Azure Kubernetes Service) and Amazon with EKS (Elastic Kubernetes Service) have jumped on to the scene in the second half of 2017.

Example:

AWS had KOPS https://kubernetes.io/docs/getting-started-guides/kops/

Azure had Azure Container Service https://docs.microsoft.com/en-us/azure/container-service/kubernetes/container-service-kubernetes-walkthrough

However, they were wrapper tools available prior to these services which would help a user create a Kubernetes cluster, but the management and the maintenance (like monitoring and upgrades) needed efforts.

Azure Container Registry:

With container demand growing, there is always a need in the market for storing and protecting the container images. Microsoft provides a Geo Replica featured private repository as a service named Azure Container Registry.

Azure Container Registry is a private registry for hosting container images. It integrates well with orchestrators like Azure Container Service, including Docker Swarm, DC/OS, and the new Azure Kubernetes service. Moreover, ACR provides capabilities such as Azure Active Directory-based authentication, webhook support, and delete operations.

The coolest feature provided is Geo-Replication. This will create multiple copies of your image and distribute it across the globe and the container when spawned will have access to the image which is nearest.

Although Microsoft has good documentation on how to set up ACR in your Azure Subscription, we did encounter some issues and hence decided to write a blog on the precautions and steps required to configure the Registry in the correct manner.

Note: We tried this using a free trial account. You can setup it up by referring the following link: https://azure.microsoft.com/en-in/offers/ms-azr-0044p/

Prerequisites:

• Make sure you have resource groups created in the supported region.
  Supported Regions: eastus, westeurope, centralus, canadacentral, canadaeast
• If you are using Azure CLI for operations please make sure you use the version: 2.0.23 or 2.0.25 (This was the latest version at the time of writing this blog)

Note: If version 2.0.24 is used it has the following bugs:
- https://stackoverflow.com/questions/48201850/login-to-the-azure-container-ser vice-fails-with-error-bool-object-has-no-attri/48204384#48204384
- https://github.com/Azure/azure-cli/issues/5300

Steps to install Azure CLI 2.0.23 or 2.0.25 (ubuntu 16.04 workstation):

Steps for Container Registry Setup:

Login to your Azure Account:

• Create a resource group:

• Create a Container Registry:

Note: SKU defines the storage available for the registry for type Basic the storage available is 10GB, 1 WebHook and the billing amount is 11 Rs/day.

For detailed information on the different SKU available, click here.

• Login to the registry :

• Sample docker file for a node application :

• Build the docker image :

• Get the login server value for your ACR :

• Tag the image with the Login Server Value:
  Note: Get the image ID from docker images command

Push the image to the Azure Container Registry:

Example:

Microsoft does provide a GUI option to create the ACR.

• List Images in the Registry:

• List tags for the Images:

• How to use the ACR image in Kubernetes deployment: Use the login Server Name + the image name

Example :

Azure Kubernetes Service

Microsoft released the public preview of Managed Kubernetes for Azure Container Service (AKS) on October 24, 2017. This service simplifies the deployment, management, and operations of Kubernetes. It features an Azure-hosted control plane, automated upgrades, self-healing, easy scaling.

Similarly to Google AKE and Amazon EKS, this new service will allow access to the nodes only and the master will be managed by Cloud Provider. For more information, you can read this.

Let’s now get our hands dirty and deploy an AKS infrastructure to play with:

• Enable AKS preview for your Azure Subscription: While AKS is in the preview, creating new clusters requires a feature flag on your subscription.

• Kubernetes Cluster Creation Command: Note: A new separate resource group should be created for the Kubernetes service. Since the service is in the preview, it is available only to certain regions.

Make sure you create a resource group under the following regions.

Example with different arguments :

Create a Kubernetes cluster with a specific version.

Create a Kubernetes cluster with a larger node pool.

• Install the Kubectl CLI :

To connect to the Kubernetes cluster from the client computer Kubectl command line client is required.

Note: If you’re using Azure CloudShell, kubectl is already installed. If you want to install it locally, run the above command:

• To configure kubectl to connect to your Kubernetes cluster :

Example :

• Verify the connection to the cluster :

• For all the command line features available for Azure check the below link: https://docs.microsoft.com/en-us/cli/azure/aks?view=azure-cli-latest

We had encountered a few issues while setting up the AKS cluster at the time of writing this blog. Listing them along with the workaround/fix:

Error: Operation failed with status: ‘Bad Request’.

Details: Required resource provider registrations Microsoft.Compute, Microsoft.Storage, Microsoft.Network are missing.

Fix: If you are using the trial account, click on subscriptions and check whether the following providers are registered or not :

• Microsoft.Compute
• Microsoft.Storage
• Microsoft.Network
• Microsoft.ContainerRegistry
• Microsoft.ContainerService

Error: We had encountered the following mentioned open issues at the time of writing this blog.

https://github.com/Azure/AKS/issues/144

https://github.com/Azure/AKS/issues/141

https://github.com/Azure/AKS/issues/145

Jenkins setup for CI/CD with ACR, AKS

Microsoft provides a solution template which will install the latest stable Jenkins version on a Linux (Ubuntu 14.04 LTS) VM along with tools and plugins configured to work with Azure. This includes:

• git for source control
• Azure Credentials plugin for connecting securely
• Azure VM Agents plugin for elastic build, test and continuous integration
• Azure Storage plugin for storing artifacts
• Azure CLI to deploy apps using scripts

Refer the below link to bring up the Instance: https://docs.microsoft.com/en-us/azure/jenkins/install-jenkins-solution-template

Pipeline plan for Spinning up a Nodejs Application using ACR — AKS — Jenkins

What the pipeline accomplishes :

Stage 1:

The code gets pushed in the Github. The Jenkins job gets triggered automatically. The Dockerfile is checked out from Github.

Stage 2:

Docker builds an image from the Dockerfile and then the image is tagged with the build number. Additionally, the latest tag is also attached to the image for the containers to use.

Stage 3:

We have default deployment and service YAML files stored on the Jenkins server. Jenkins makes a copy of the default YAML files, make the necessary changes according to the build and put them in a separate folder.

Stage 4:

kubectl was initially configured at the time of setting up AKS on the Jenkins server. The YAML files are fed to the kubectl util which in turn creates pods and services.

Sample Jenkins pipeline code :

What we achieved:

• We managed to create a private Docker registry on Azure using the ACR feature using az-cli 2.0.25.
• Secondly, we were able to spin up a private Kubernetes cluster on Azure with 2 nodes.
• Setup Up Jenkins using a pre-cooked template which had all the plugins necessary for communication with ACR and AKS.
• Implement a small Continuous Deployment pipeline in Jenkins which used all the Docker features.

*****************************************************************

This post was originally published on Velotio Blog.

Velotio Technologies is an outsourced software product development partner for technology startups and enterprises. We specialize in enterprise B2B and SaaS product development with a focus on artificial intelligence and machine learning, DevOps, and test engineering. We combine innovative ideas with business expertise and cutting-edge technology to drive business success for our customers.

Interested in learning more about us? We would love to connect with you on ourWebsite, LinkedIn or Twitter.

*****************************************************************