Bozobooks.com: Fullstack k8s application blog series

GitOps Architecture with GitHub Actions & ArgoCD with Slack Notifications

Chapter 4: In this blog, we will build the GitOps infrastructure for the project. We will be using GitHub actions for the continuous integration and ArgoCD for deploying the application to the Kubernetes cluster. We will also be integrating the pipeline with Slack, to notify the status.

GitOps Architecture

In Chapter 3, we build a Quarkus application, as you would have gone through the build and deploy process, there are a lot of steps, that we need to execute manually. It is always the best practice to automate the build and deployment (Continous Integration/Continous Deployment).

In this blog, we will go through the architecture and how we will organize the code. As we all know, on K8s and the cloud, Infrastructure is also code. So we have an Application Code, Infrastructure Code. We will be using the GitOps principles to build and deploy the application & Infrastructure.

The following picture shows the high-level GitOps architecture and flow.

1. The developer pushes the code to the GitHub application repository.
2. This triggers the GitHub Action workflow. The workflow builds the application, creates a native image, and performs basic testing
3. The workflow then builds a Docker Image and pushes it to DockerHub, with the right build number and tagging.
4. The Docker image details are passed to the Infrastructure repository workflow.
5. This will trigger the Infrastructure Workflow will then pick the Docker image details, and update the Infrastructure code
6. Slack is integrated to continuously pass the status of the GitOps workflow so that the engineer can get notifications of success/failure
7. ArgoCD is installed in the respective Kubernetes clusters, listening to respective Kubernetes Infrastructure code (manifests), and this will trigger a deployment automatically.

GitHub Repository Design

To have this workflow run across application and infrastructure code, we will have the plan to organize our code. The following picture illustrates the Git repository structure.

Application Code Repository: Application code is stored in 3 folders

• bozo-book-info-service: This has the application code (Quarkus) for BookInformationService MicroService, which fetches and caches book information from Google Book API.
• bozo-library-service: This contains the application (Quarkus) for BookLibraryService MicroService that stores and provides personal Book preferences.
• bozo-book-library-ui: This contains the SPA (Single page application, built using ReactJS). This will be the front end of the Bozo Book Library application

Infrastructure Code: Infrastructure code is stored in separate folders, based on the environment. For this project, we will have dev (in Docker Desktop/Rancher Desktop) environment and prod (in AWS). (In this blog we will focus on the dev environment, and we will be covering the prod in future blogs). For each of the environments, we will have the following two folders.

• helm-terraform: This will have the helm code for all dependencies (Vault, Prometheus, Grafana, Loki, OpenTelemetry, Zipkin, Ingress, cert-manager etc), and the terraform code to deploy these helm charts.
• bozobook-manifests: This will have all the k8s manifest files for the application. (We will be moving these k8s manifests to helm in future blogs)

The respective repository you can look at these repositories (the code is a work in progress :-D so stay patient till I finish this series, to have the final code)

• https://github.com/abvijaykumar/book-library-kube-app
• https://github.com/abvijaykumar/book-library-kube-infra

Let's now set up the base infrastructure for GitOps

Install & Configure ArgoCD

Let's start with installing ArgoCD in our Dev Cluster. Execute the following commands

kubectl create namespace argocd
kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml

Once we see the Pods are up and running, we can get the initial password by executing the following command.

kubectl -n argocd get secret argocd-initial-admin-secret -o jsonpath="{.data.password}" | base64 -d

Once we get the password, we can access the ArgoCD UI by creating a port forward using kubectl port-forward for the argocd-server service. The following shows the initial landing page for ArgoCD UI.

Let's create an application for the Dev environment. The following YAML, shows the various parameters, that we need to configure.

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
   name: bozobooks-dev
spec:
  destination:
    name: ''
    namespace: bozo-book-library-dev
    server: 'https://kubernetes.default.svc'
  source:
    path: dev/bozobooks-manifests
    repoURL: 'https://github.com/abvijaykumar/book-library-kube-infra'
    targetRevision: HEAD
    directory:
       recurse: true
  project: default
  syncPolicy:
    automated:
      prune: false
      selfHeal: false
    retry:
      limit: 2
      backoff:
        duration: 5s
        maxDuration: 3m0s
        factor: 2

This code is self-explanatory, which shows which Git repository ArgoCD is listening, to and where it shud deploy the infrastructure code. The following screenshot shows the code configured on ArgoCD UI.

Once we submit, we should see ArgoCD, pull all the Kubernetes manifests from the Git repository and execute. The following screenshot shows the deployment status of the cluster and the topology.

We have the CD setup, now let's set up the CI with GitHub Actions. (For basic information on GitHub Actions, please refer to this site)

CI GitHub Actions

Before we start writing the workflow code, let's set up the environment. Go to your GitHub repository and set the following secrets by navigating to Settings->Secret->Actions. You will see something like the following screen

We need the following variables to be set

• API_TOKEN_GITHUB — This is the API token that we need to call the Infrastructure code. (Refer to this site for details on how to create an access token).
• DOCKERHUB_USERNAME, DOCKERHUB_PASSWROD — These are the login credentials to your DockerHub account.
• GITOPS_SLACK_API — This is the Slack Webhook URL. Let's see how to create a slack webhook in the next section

Slack Integration

To integrate with Slack, we need to create a Slack app and create a webhook. You can find it here https://api.slack.com/messaging/webhooks

I created an app named Bozobooks, in a workspace I created. Once created, you should go to the “Incoming Webhooks” option, as shown in the screenshot below

and activate it, but click on the switch, as shown in the below screenshot

scroll down and you will find the “Add New Webhook to Workspace” button, click on it and a new Webhook URL is generated.

After this select the respective channel, where you want the messages to go. I created a channel called “gitops” and selected that. This will generate a Webhook URL. Store it safely. We will be saving it in the GitHub secrets, to be accessed in our GitHub actions workflows.

We will be using ravsamhq/notify-slack-action@v2 GitHub Actions plugin to notify on slack. the following is a sample code. We can pass the status, token, title, message, and footer to this plugin.

- name: Starting Build & Push  Status Slack Notification
  uses: ravsamhq/notify-slack-action@v2
  if: always()
  with:
     status: ${{ job.status }}
     token: ${{ secrets.GITHUB_TOKEN }}
     notification_title: "Starting Build {BUILD_NUMBER}"
     message_format: "{emoji} *{workflow}* {status_message} in <{repo_url}|{repo}>"
     footer: "Linked Repo <{repo_url}|{repo}> | <{workflow_url}|View Workflow>"
     notify_when: "failure, success, warning"
   env:
     SLACK_WEBHOOK_URL: ${{ secrets.GITOPS_SLACK_API }}

Generating Build number

To generate build numbers we will use einaregilsson/build-number@v3 plugin, here is the sample code.

- name: Generate build number
  uses: einaregilsson/build-number@v3
  with:
    token: ${{secrets.github_token}}

Passing the Build number to the Infrastructure code

In the application workflow, We have a requirement to invoke Infrastructure workflows. To do that, we will need to do some setup.

The Infrastructure code workflow needs to be coded to trigger for repository_dispatch. Here is the sample code

on:
   repository_dispatch:
      types: deploy-book-info-service-infra-updated-build

This can then be invoked with the following code from the application workflow

- name: Invoke book-info-service Infra GitOps pipe
  run: |
     curl -H "Accept: application/vnd.github.everest-preview+json" \
     -H "Authorization: token ${{secrets.API_TOKEN_GITHUB}}" \
     --request POST \
     --data '{"event_type": \
     "deploy-book-info-service-infra-updated-build",\         
     "client_payload": { "buildnumber":\
     "'"$BUILD_NUMBER"'" }}'\   https://api.github.com/repos/abvijaykumar/book-library-kube-infra/dispatches

We will be invoking the dispatch (infrastructure code workflow) using curl, with the following header in the HTTP POST request

• Authorization: token-based with token, that we stored in GitHub secrets API_TOKEN_GITHUB
• event_type: name of the type, we defined as repository_dispatch
• client_payload: JSON that contains the build number, that we will be using in the Infrastructure code workflow, to update our YAML

We will be covering this in the next chapter, where we will walk through the complete GitHub actions code for deploying BozoBookInformationService.

Hope this was helpful, See you in the next chapter

until then stay safe, and have fun…

Please leave your feedback and comments.

Further Reading:

Kubernetes Monitoring with OpenTelemetry

Top 11 Tools for Microservices Backend Development in 2023

Deploying OpenTelemetry with Java: Your Guide

5 Microservices Challenges and Blindspots for Developers

Developers vs. DevOps - the case for developer ownership