Starting with AZURE DEVOPS 🚀

Building a Basic CI/CD Pipeline (Part 2) 🏗️

Follow-Up Introduction

Welcome back to the second part of my blog series on building a CI/CD pipeline using Azure DevOps! In the first part, we explored how to set up a robust Continuous Integration (CI) pipeline. Now, we’ll dive into Continuous Delivery (CD), where we’ll automate the deployment process to ensure that every change is always in a deployable state. By the end of this guide, you’ll have a fully functional CI/CD pipeline leveraging Kubernetes and ArgoCD for seamless deployments.

Part 2: Continuous Delivery

1. Go to Microsoft Azure -> Kubernetes Service

First, navigate to the Microsoft Azure portal and go to the Kubernetes Service section.

Step-by-Step:

1. Sign in to the Azure Portal.
2. In the left-hand menu, select “Kubernetes services.”
3. Click “Create” to create a new Kubernetes service.

2. Create Kubernetes Cluster

Next, create a Kubernetes cluster to host your applications. Fill in the required fields like cluster name, region, and Kubernetes version.
Click “Next”.

3. Update Node Pool

Update the node pool in your Kubernetes cluster to ensure it meets your application’s resource requirements.

Step-by-Step:

1. Navigate to your Kubernetes cluster.
2. Go to “Node pools” and select your node pool.
3. Update the size and scaling options as needed.

Why GitOps?

GitOps is an operational framework that uses Git repositories as the source of truth for declarative infrastructure and applications. It enables continuous reconciliation, where the actual state of your cluster is continuously compared with the desired state defined in Git. If drift is detected, it is automatically corrected.

5. Kubernetes Deployed

With your cluster set up, Kubernetes is ready to deploy and manage your applications.

6. Login to Azure CLI

Log in to Azure CLI to interact with your Kubernetes cluster.

Step-by-Step:

1. Download Azure CLI.
2. Open your terminal.
3. Run az login to sign in to your Azure account.
4. Use az account set --subscription <your-subscription-id> to set the desired subscription.

7. Install ArgoCD

Install ArgoCD, a declarative, GitOps continuous delivery tool for Kubernetes. Run the following commands:

kubectl create namespace argocd
kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml

8. Why Use ArgoCD?

ArgoCD provides a powerful toolset for managing Kubernetes deployments through GitOps practices. It offers features like automated sync, health checks, and rollback capabilities, making it easier to manage complex Kubernetes applications.

9. Retrieving Password to Access ArgoCD UI on Web

Retrieve the password to access the ArgoCD UI.

kubectl get pods -n argocd
kubectl edit secret argocd-initial-admin-secret -n argocd

10. Convert Base64 to Plaintext

Convert the base64 encoded password to plaintext using this command.

11. Finding Port and IP to Access ArgoCD on Web

Find the port and IP address to access ArgoCD.

kubectl edit svc argocd-server -n argocd

Replace ClusterIP to NodePort in argocd-server

#port
kubectl get svc -n argocd

#external-ip
kubectl get nodes -o wide

Here, port number is 31347

Here, PublicIP is 52.190.0.62

12. Edit Inbound Port on Microsoft Azure’s Virtual Machine Scale Set to Access Port

Edit the inbound port rules to allow access to the ArgoCD UI.

Step-by-Step:

1. Navigate to your VM scale set in Azure.
2. Go to “Networking” and add an inbound port rule to allow traffic on the required port.

13. Login to Argo

Access the ArgoCD UI using the retrieved IP address and port, and log in with the “admin” credentials.

14. Connect Repo to Azure DevOps

Connect your Git repository to Azure DevOps.

Step-by-Step:

1. In ArgoCD, go to “Settings” -> “Repositories.”
2. Click “Connect Repo” and enter your repository details.

15. Use Personal Access Token in Repo URL

Use a personal access token (PAT) for authentication when connecting the repository.

Step-by-Step:

1. Generate a PAT in your azure devops.
2. Use the PAT in the repository URL format: https://<PAT>@dev.azure.com/<repo>

16. Create a Script Folder with File UpdateK8sManifest.sh in Repo in Azure DevOps

Create a script in your repository to update Kubernetes manifests.

• Why We Need It?: This script automates the process of updating Kubernetes manifests, ensuring that your cluster’s desired state is always in sync with the repository.

#!/bin/bash

set -x

# Set the repository URL
REPO_URL="https://<ACCESS-TOKEN>@dev.azure.com/<AZURE-DEVOPS-ORG-NAME>/voting-app/_git/voting-app"

# Clone the git repository into the /tmp directory
git clone "$REPO_URL" /tmp/temp_repo

# Navigate into the cloned repository directory
cd /tmp/temp_repo

# Make changes to the Kubernetes manifest file(s)
# For example, let's say you want to change the image tag in a deployment.yaml file
sed -i "s|image:.*|image: <ACR-REGISTRY-NAME>.azurecr.io/$2:$3|g" k8s-specifications/$1-deployment.yaml

# Add the modified files
git add .

# Commit the changes
git commit -m "Update Kubernetes manifest"

# Push the changes back to the repository
git push

# Cleanup: remove the temporary directory
rm -rf /tmp/temp_repo

17. Edit Pipeline to Add “Update” Stage

Edit your Azure DevOps pipeline to include an “update” stage that runs the script.

If you’re encountering errors which are due to the presence of Windows-style carriage return (\r) characters in your shell script, this can cause issues when running the script in a Unix-based environment like Azure DevOps.

Well if you didn’t, to fix this, you need to convert your shell script from Windows (CRLF) line endings to Unix (LF) line endings. You can do this:

# Docker
# Build and push an image to Azure Container Registry
# https://docs.microsoft.com/azure/devops/pipelines/languages/docker

trigger:
 paths:
   include:
     - vote/*

resources:
- repo: self

variables:
  # Container registry service connection established during pipeline creation
  dockerRegistryServiceConnection: '<registry>'
  imageRepository: 'votingapp'
  containerRegistry: '<container-name>.azurecr.io'
  dockerfilePath: '$(Build.SourcesDirectory)/result/Dockerfile'
  tag: '$(Build.BuildId)'

pool:
 name: 'azureagent'

stages:
- stage: Build
  displayName: Build
  jobs:
  - job: Build
    displayName: Build
    steps:
    - task: Docker@2
      displayName: Build an image to container registry
      inputs:
        containerRegistry: '$(dockerRegistryServiceConnection)'
        repository: '$(imageRepository)'
        command: 'build'
        Dockerfile: 'result/Dockerfile'
        tags: '$(tag)'

- stage: Push
  displayName: Push
  jobs:
  - job: Push
    displayName: Push
    steps:
    - task: Docker@2
      displayName: Push an image to container registry
      inputs:
        containerRegistry: '$(dockerRegistryServiceConnection)'
        repository: '$(imageRepository)'
        command: 'push'
        tags: '$(tag)'

- stage: Update
  displayName: Update
  jobs:
  - job: Update
    displayName: Update
    steps:
    - script: |
       sudo apt-get update
       sudo apt-get install -y dos2unix
       dos2unix scripts/UpdateK8sManifest.sh
      displayName: Convert line endings to Unix (LF)
    - task: ShellScript@2
      inputs:
        scriptPath: 'scripts/UpdateK8sManifest.sh'
        args: 'vote $(imageRepository) $(tag)'

18. Run Pipeline

Run the pipeline. If it is successful but not updating the application, you may need to create a Kubernetes secret for image pulling.

19. Create Key

Create a Kubernetes secret with the following command:

kubectl create secret docker-registry <secret-name> \
    --namespace <namespace> \
    --docker-server=<container-registry-name>.azurecr.io \
    --docker-username=<service-principal-ID> \
    --docker-password=<service-principal-password>

20. Add imagePullSecrets in YAML Code

Add imagePullSecrets to your Kubernetes deployment YAML file.

Run the pipeline again.

21. Success, Argo’ status is Healthy and Synced

If everything is configured correctly, ArgoCD should show your application as healthy and synced.

22. Find the Portal and Add to Inbound Network

Find the portal IP and add it to the inbound network rules to access the application.

kubectl get svc
kubectl get nodes -o wide

23. Access the Application

23. Do the Same for Worker and Result Node if You Want to Go Further

Repeat the above steps for additional nodes such as worker and result nodes to ensure that all components of your application are deployed and managed consistently.

Conclusion

Congratulations on completing the setup of your Continuous Delivery (CD) pipeline with Azure DevOps, Kubernetes, and ArgoCD! This journey has taken us through the essential steps of deploying and managing applications in a streamlined and automated manner. By integrating these powerful tools, you’ve ensured that every code change is automatically tested, built, and deployed, enabling faster and more reliable delivery of software.

In the first part of this series, we established the Continuous Integration (CI) pipeline, ensuring that code changes are automatically tested and built. In this second part, we extended our pipeline to handle Continuous Delivery (CD), automating the deployment process using Kubernetes and ArgoCD. We’ve covered:

1. Creating and configuring a Kubernetes cluster in Azure.
2. Setting up GitOps for continuous reconciliation and drift detection.
3. Installing and configuring ArgoCD for managing Kubernetes deployments.
4. Connecting Azure DevOps with your Git repository and Kubernetes cluster.
5. Automating the update process for Kubernetes manifests.
6. Creating necessary secrets and configuring the deployment pipeline.

By following these steps, you’ve built a robust CI/CD pipeline that not only automates the integration and deployment processes but also ensures that your applications are always in a deployable state. This setup reduces manual intervention, minimizes the risk of human error, and accelerates the overall development lifecycle.

As you continue to refine and expand your pipeline, consider the following best practices:

• Monitor and Optimize: Continuously monitor your pipeline’s performance and optimize as needed. Look for bottlenecks and areas for improvement.
• Security: Implement security best practices, such as scanning for vulnerabilities, managing secrets securely, and ensuring proper access controls.
• Scalability: Design your pipeline to handle increased workloads and scale as your project grows.
• Documentation: Maintain comprehensive documentation for your pipeline setup and processes to facilitate onboarding and troubleshooting.

Thank you for following along with this blog series. I hope it has provided valuable insights and practical steps to enhance your DevOps practices. Stay tuned for more tutorials and updates as we continue to explore the world of DevOps and cloud computing.

Happy coding and deploying!

Connect to me, if you have any query, want to share or suggest me anything, here’s my Portfolio or dm me on LinkedIn. Stay tuned and follow me for more chapters of my odyssey, my projects, cybersec and devops articles, and write-ups and also thank you for being part of my story! 🚀

If you enjoy my posts and find them helpful, consider buying me a coffee! Your support can help me to earn relevant certifications and continue sharing valuable insights. Buy Me a Coffee

https://buymeacoffee.com/harshit21

Let’s continue to strengthen our skills and build secure, scalable applications together!