GitOps & IaC: Deploying GKE clusters using Terraform & Google Cloud build.

IaC and GitOps are well known concepts in the DevOps/platform engineering space, but how can these concepts be applied within the GCP ecosystem? This blog attempts to give insights on how to set up an IaC delivery pipeline.

Let’s explore the delivery architecture

With Git as the central source of truth, a delivery-pipeline project drives infra provisioning across multiple gcp projects in this case Dev, UAT & Production. Terraform is leveraged for IaC.

Set up summary:

• Create a GCS bucket required as a backend to store your terraform state.
• Create three (3) git repositories dev-cluster, uat-cluster & prod-cluster. We can also use a single repo and map environments to folders within the environment directory.
• Create four (4) projects a delivery-pipeline, dev-cluster, uat-cluster & prod-cluster projects in GCP.
• Enable Cloud Build API in the delivery-pipeline project & Kubernetes Engine API in the dev-cluster, uat-cluster & prod-cluster projects.
• Create Dev, UAT & Prod cloud build triggers in the delivery-pipeline project with the corresponding git repo mapping.
• Grant the cloud build service account of the delivery-pipeline project permission to provision resources in your dev-cluster, uat-cluster & prod-cluster projects
• Run dev, uat and prod cloud build triggers in the delivery-pipeline project.

Let’s get started!!!!

Step 1: Create a GCS bucket required for your terraform state backend

gcloud config set project PROJECT_ID
gcloud storage buckets create gs://BUCKET_NAME

Step 2: Create three (3) git repositories dev-cluster, uat-cluster & prod-cluster

Clone the repository https://github.com/Yougo007/gke-cluster.git

As part of this setup, you would need to create three git branches dev-cluster, uat-cluster & prod-cluster branches as shown below. How to create git branches can be found here.

root@penguin:/home/yougo/gke-cluster# git branch
 dev-cluster
* master
 prod-cluster
 uat-cluster

Navigate to your version of https://github.com/Yougo007/gke-cluster/blob/dev-cluster/environments/dev/backend.tf and update it with your newly created gcs backend bucket.

Let’s explore the repo briefly!!!! It consist of three components; README, cloudbuild.yaml & an environments folder.

root@penguin:/home/yougo/gke-cluster# ls -lrt
total 8
-rw-r--r-- 1 root root  261 Dec 13 10:12 README
-rw-r--r-- 1 root root 2809 Dec 15 01:04 cloudbuild.yaml
drwxr-xr-x 1 root root   20 Dec 15 01:05 environments

I. A cloudbuild.yaml. A build config file contains instructions for Cloud Build to perform tasks based on your specifications. For example, your build config file can contain instructions to build, package, and push Docker images.

This page explains the schema of the Cloud Build configuration file. For instructions on creating and using a build config file, see Creating a basic build config file.

More on cloud build configuration can be found here

steps:
- id: 'Fetching Branch Name'
  name: 'alpine'
  entrypoint: 'sh'
  args: 
  - '-c'
  - | 
      echo "***********************"
      echo "$BRANCH_NAME"
      echo "***********************"
- id: 'Building DEV cluster'
  name: 'hashicorp/terraform:0.14.6'
  entrypoint: 'sh'
  args: 
  - '-c'
  - |
      cd environments/dev && terraform init && terraform plan && terraform apply -auto-approve
timeout: '3600s'
options:
  machineType: 'N1_HIGHCPU_8'

II. An environments folder. Each environment folder contains corresponding terraform templates that map to each branch & an upstream GKE environment.

III. A git README file.

Step 3: Create four (4) projects a delivery-pipeline, dev-cluster, uat-cluster & prod-cluster projects in GCP.

Open cloudshell and run the following commands; You can also create these from the gcp console.

gcloud projects create - name="delivery-pipeline" - labels=type=cicd
gcloud projects create - name="dev-cluster" - labels=type=dev
gcloud projects create - name="uat-cluster" - labels=type=uat
gcloud projects create - name="prod-cluster" - labels=type=prod

Step 4: Enable Cloud Build API in the delivery-pipeline project & Kubernetes Engine API in the dev-cluster, uat-cluster & prod-cluster projects

delivery-pipeline project

gcloud config set project YOUR_PROJECT_ID
gcloud services enable cloudbuild.googleapis.com
dev-cluster, uat-cluster & prod-cluster projects
gcloud config set project YOUR_PROJECT_ID
gcloud services enable container.googleapis.com

Step 5: Create Dev, UAT & Prod cloud build triggers in the delivery-pipeline project with the corresponding git repo mapping.

Repeat this for uat & prod git repo.

Step 6: Grant the cloud build service account of the delivery-pipeline project permission to provision resources in your dev, uat & prod cluster projects

Copy the service account from the delivery-pipeline cloud build project and follow the steps below.

• Navigate to IAM & Admin → Service accounts in the dev-cluster project

• Grant Editor access to the delivery-pipeline service account. Repeat this process for the uat-cluster project & prod-cluster project

Step 7: Run the dev, uat and prod build triggers in the delivery-pipeline project

Voila!!! Now we have a fully functional IaC pipeline for GKE provisioning. In my next blog we would leverage this pipelines for solution deployment.

Clean Up to avoid unexpected bills

Please navigate to each project and delete the provisioned GKE clusters. You can also create an infra destroy cloud build trigger.