How to apply GitOps CD with GKE and Cloud-build in 5 minutes!
Effective engineers will always prefer high leverage work to get done first, high leverage work is a kind of time investment, it’s to spend some time and efforts to build something that will save a lot of time in the future or prevent many human errors, one of these good investments is CI/CD pipelines.
CI/CD pipelines have never been a need only for big teams, even a single developer will need it, whatever the project it’s, however, it could have a few or many steps based on team size and needs.
Kubernetes and Cloud-native applications
I’m gonna assume that your application is a Cloud-native application built on top of microservices, each microservice is built by a different programming language and it’s communicating with other microservices in an organized way.
When we talk about Cloud-native applications and microservices probably means we use containers and containers orchestration (Kubernetes), I won’t talk about all benefits of Kubernetes in this article, but I would mention the greatness of IAC (Infrastructure as a code) when there is a code describes the current state of your infrastructure, so you can move it easily or rebuild again in seconds/minutes.
What is GitOps?
GitOps is a way of implementing Continuous Deployment for cloud native applications. It focuses on a developer-centric experience when operating infrastructure, by using tools developers are already familiar with, including Git and Continuous Deployment tools. — https://www.gitops.tech/
The main idea is to have a Git Repo (Environment Repo) that hosts the current state of infrastructure (Kubernetes’ manifests), then changing application on the cloud would be coming from changes done on the Environment repo after it got sync with Kubernetes cluster.
Pros of GitOps
- Deploy faster on the cloud, no need to configure your local machine, useful if you manage multiple environments
- Since it’s Git-based, only revert the last commit to recovery!
- Safe, no need to store credentials for infrastructure on local machines
- Use Git permissions to know who can read and write
- Use Git history to know what happened and when it happened
Methods of GitOps
After preparing the Environment repo and putting all required manifests to run the applications on Kubernetes we have to sync these manifests with the Kubernetes cluster, which means any changes that happen on the Environment repo should get reflect on the current running infrastructure.
- Pushed-based updates: changes on the Environment repo will trigger a job to update the current Kubernetes state to follow the new one in the Environment repo.
- Pull-based updates: Kubernetes cluster will run a job periodically to check/observe the current state described in the Environment repo and update itself if needed.
GitOps example
My simple example would be about having a web application repo and the environment repo, I use GKE to manage my Kubernetes cluster, we need two Cloud build triggers here:
- Application repo trigger: which is gonna build a new docker image, push it to container registry / Docker hub, last but not least, it will update the environment repo to use the last web application image instead of the old one.
- The environment repo trigger: which is gonna run when there is a new commit to apply the latest changes on the Kubernetes cluster.
The environment repo will look like this:
I’m using kustomize, it will make things easier in changing parts of Kubernetes objects, for example, if you have many cronjobs and all of them using the same image and you need to use a newer one, you’ve to update many YAML files and apply all of them, but if you’re using kustomize, you update and apply one file and that will update all cronjobs! very nice!.
So here are the required changes to be implemented in Application Repo (Demo API)
This cloudbuild.yaml should be added to the top level of the Application repo.
ENV_REPO_FILES: These are the paths of kustomization files you want to update in the environment repo.
I used a custom step omegaes/kubci as POC how to build a custom step for Cloud-build, this step will do three tasks:
- Clone the environment repo, checkout target branch to do required changes
- Update Kustomiztion file, change the image tag of the application’s deployment
- Commit these changes and merge them with the main branch in the Environment repo.
* It’s better to break the build tasks into multiple steps.
* It’s better to let app pipeline to create a target branch from main branch in the Environment repo, another trigger on target branch will check if it’s possible to apply these changes, if it’s okay it will apply and merge to main branch on the Environment repo.
This cloudbuild.yaml should be added to the top level of your environment repo.
Finally, create two Cloud-build triggers and the secret “Environment-repo-auth-URL”:
“Environment-repo-auth-URL” will be used in the custom step previously to clone the Environment Repo and push a new image tag there.
Let’s make a small change on Application Repo:
Conclusion
What we’ve gone through was only a simple example for GitOps, in real-world projects you will find many struggles, this approach was good for me in controlling multiple Kubernetes clusters from the Environment Git repository, please don’t hesitate to ask for any clarification or to share your thoughts!!
You can check out these Github repositories to see how I build the demo here:
- GKE Demo App (Hello World in node.js)
- GKE-Demo-Environment (host Kubernetes objects manifests)
- Kubci (A Docker image meant to used as a custom step in Cloud build, this will clone the environment repo, update kustomization yaml and push these changes back to the environment repo)
Resources I was inspired by:
