GitOps: In a nutshell

Introduction

In today’s fast-paced cloud environment, Platform Engineering, and DevOps teams face increasing pressure to deliver reliable, scalable, and maintainable Kubernetes infrastructures. GitOps has emerged as the gold standard for managing these complex systems, offering a declarative approach that leverages Git as the single source of truth for both infrastructure and application deployments.

At the heart of many GitOps implementations is ArgoCD, a powerful continuous delivery tool for Kubernetes. When strategically integrated with infrastructure-as-code tools like Terraform and package managers like Helm, along with customization tools like Kustomize, platform teams can build robust deployment pipelines that dramatically simplify operational workflows.

This post explores how to set up and manage ArgoCD using Terraform in a Google Cloud Platform (GCP) environment, and how to integrate Helm and Kustomize into this workflow. By the end of this guide, you’ll understand how to:

1. Install ArgoCD via Terraform
2. Structure your repositories for optimal GitOps workflows
3. Integrate Helm and Kustomize with ArgoCD
4. Manage your entire application deployment pipeline using these tools

Photo by krakenimages on Unsplash

This solution blog is a collaboration of Rahul Kumar Singh (Senior Architect at EY and Google Developer Experts(GDE) for Google Cloud) and Rohan Singh (Senior Cloud Infrastructure Engineer and Google Developer Experts(GDE) for Google Cloud) at SADA Systems — An Insight Company.

Prerequisites

Before we dive in, ensure you have the following:

• A running Google Kubernetes Engine (GKE) cluster
• Basic knowledge of ArgoCD
• Understanding of Terraform, Helm, and Kustomize
• Google Cloud SDK configured
• kubectl installed and configured
• Git repositories set up (we’ll discuss the structure in detail)

Why This Approach Matters for Platform Teams

The integrated approach provides substantial advantages for platform engineering and DevOps teams. Let’s explore these benefits before delving into the technical specifics.

1. Standardization and Consistency

Standardizing the deployment process across all environments and applications eliminates the common issue of “it works in my environment”. This can be achieved by managing ArgoCD with Terraform and integrating with Helm and Kustomize.

2. Reduced Operational Overhead

Establishing this pipeline streamlines application management. Consequently, platform engineers can focus on infrastructure improvements instead of manual deployments.

3. Improved Security and Compliance

With everything defined as code and stored in Git, you gain comprehensive audit trails, approval workflows via pull requests, and the ability to enforce security policies across your entire infrastructure.

4. Faster Onboarding

New team members can quickly understand the entire system by exploring the code repositories, rather than relying on tribal knowledge or outdated documentation.

5. Scalable Team Structure

This approach divides responsibilities between platform teams (infrastructure and deployment) and application teams (application code), allowing both to operate efficiently without conflict.

Repository Structure: A Best Practice Approach

For our GitOps implementation, we’ll adhere to the practice of using three separate and dedicated repositories — a pattern that has proven successful for almost all organizations of all sizes:

1. IaC Repo: Contains Terraform code for ArgoCD installation and other infrastructure resources

• Owned by: Platform/Infrastructure teams
• Change frequency: Low to medium

2. DevOps Repo: Contains Kustomize configurations, ArgoCD application definitions, and other operational resources

• Owned by: Platform/DevOps teams
• Change frequency: Medium

3. Application Repo: Contains application-specific files, including Helm values.yaml files

• Owned by: Application development teams
• Change frequency: High

This clear separation of concerns creates a structured workflow where each team can operate independently within their domain while maintaining integration across the entire system. It also aligns perfectly with the principle of least privilege — developers can modify application configurations without needing access to infrastructure code.

Installing ArgoCD via Terraform

We’ll use Terraform with the kubernetes_manifest resource to install ArgoCD in our GKE cluster. Let’s start with the basic structure of our Terraform code in the IaC repo.

Directory Structure for IaC Repo

iac-repo/
├── main.tf
├── variables.tf
├── outputs.tf
├── providers.tf
└── modules/
   └── argocd/
       ├── main.tf
       ├── variables.tf
       ├── outputs.tf
       └── manifests/
           ├── namespace.yaml
           ├── install.yaml
           └── ingress.yaml

Setting Up Providers

First, let’s set up our providers in providers.tf:

terraform {
 required_providers {
   google = {
     source  = "hashicorp/google"
     version = "~> 4.0"
   }
   kubernetes = {
     source  = "hashicorp/kubernetes"
     version = "~> 2.10"
   }
 }
}

provider "google" {
  project = var.project_id
  region  = var.region
}
provider "kubernetes" {
  host                   = "https://${data.google_container_cluster.my_cluster.endpoint}"
  token                  = data.google_client_config.default.access_token
  cluster_ca_certificate = base64decode(data.google_container_cluster.my_cluster.master_auth[0].cluster_ca_certificate)
}
data "google_client_config" "default" {}
data "google_container_cluster" "my_cluster" {
  name     = var.cluster_name
  location = var.cluster_location
}

ArgoCD Module

Now, let’s create an ArgoCD module to encapsulate the installation. Here’s what our modules/argocd/main.tf file would look like:

resource "kubernetes_namespace" "argocd" {
 metadata {
   name = "argocd"
 }
}

resource "kubernetes_manifest" "argocd_install" {
  for_each = fileset("${path.module}/manifests", "install.yaml")
  
  manifest = yamldecode(file("${path.module}/manifests/${each.value}"))
  
  depends_on = [kubernetes_namespace.argocd]
}
resource "kubernetes_manifest" "argocd_ingress" {
  manifest = yamldecode(file("${path.module}/manifests/ingress.yaml"))
  
  depends_on = [kubernetes_manifest.argocd_install]
}
# Extract ArgoCD admin password
data "kubernetes_secret" "argocd_admin_password" {
  metadata {
    name      = "argocd-initial-admin-secret"
    namespace = "argocd"
  }
  
  depends_on = [kubernetes_manifest.argocd_install]
}
# Output the password for use later
output "argocd_admin_password" {
  value     = data.kubernetes_secret.argocd_admin_password.data.password
  sensitive = true
}

We need to create our ArgoCD installation manifests. You can download the official ArgoCD manifest and save it to modules/argocd/manifests/install.yaml. For our purposes, we’ll use the latest stable version.

Main Terraform Configuration

Now, let’s use this module in our main configuration:

module "argocd" {
  source = "./modules/argocd"
  
  # Pass any necessary variables
}

# Output the ArgoCD admin password
output "argocd_password" {
  value     = module.argocd.argocd_admin_password
  sensitive = true
}
# Optionally, output the ArgoCD server URL
output "argocd_server_url" {
  value = "https://argocd.${var.domain}"
}

Setting Up the DevOps Repository

With ArgoCD installed, let’s set up our DevOps repository to define our applications and deployment configurations.

Directory Structure for DevOps Repo

devops-repo/
├── applications/
│   ├── my-app/
│   │   ├── base/
│   │   │   └── application.yaml
│   │   ├── overlays/
│   │   │   ├── dev/
│   │   │   │   ├── kustomization.yaml
│   │   │   │   └── patches/
│   │   │   │       └── application.yaml
│   │   │   └── prod/
│   │   │       ├── kustomization.yaml
│   │   │       └── patches/
│   │   │           └── application.yaml
│   │   └── kustomization.yaml
│   └── kustomization.yaml
└── projects/
    ├── base/
    │   └── project.yaml
    ├── overlays/
    │   ├── dev/
    │   │   ├── kustomization.yaml
    │   │   └── patches/
    │   │       └── project.yaml
    │   └── prod/
    │       ├── kustomization.yaml
    │       └── patches/
    │           └── project.yaml
    └── kustomization.yaml

Defining ArgoCD Applications with Kustomize

Let’s create an ArgoCD Application definition in applications/my-app/base/application.yaml:

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: my-application
  namespace: argocd
spec:
  project: default
  source:
    repoURL: https://github.com/your-org/application-repo.git
    targetRevision: HEAD
    path: helm-charts/my-app
    helm:
      valueFiles:
        - values.yaml
  destination:
    server: https://kubernetes.default.svc
    namespace: my-app
  syncPolicy:
    automated:
      prune: true
      selfHeal: true

Next, let’s create Kustomize overlays for different environments. Here’s what applications/my-app/overlays/dev/kustomization.yaml might look like:

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

resources:
  - ../../base
patches:
  - path: patches/application.yaml

And applications/my-app/overlays/dev/patches/application.yaml:

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: my-application
spec:
  source:
    targetRevision: dev
    helm:
      valueFiles:
        - values-dev.yaml
  destination:
    namespace: my-app-dev

Similar files would be created for the production environment.

Application Repository with Helm Charts

Finally, let’s set up our application repository, which contains our Helm charts and values files.

Directory Structure for Application Repo

pplication-repo/
└── helm-charts/
    └── my-app/
        ├── Chart.yaml
        ├── templates/
        │   ├── deployment.yaml
        │   ├── service.yaml
        │   └── ingress.yaml
        ├── values.yaml
        ├── values-dev.yaml
        └── values-prod.yaml

Sample values.yaml

Here’s an example of a base values.yaml:

image:
  repository: gcr.io/my-project/my-app
  tag: latest

replicaCount: 2
resources:
  requests:
    memory: "256Mi"
    cpu: "100m"
  limits:
    memory: "512Mi"
    cpu: "500m"
service:
  type: ClusterIP
  port: 80

And the environment-specific values-dev.yaml:

replicaCount: 1

resources:
  requests:
    memory: "128Mi"
    cpu: "50m"
  limits:
    memory: "256Mi"
    cpu: "250m"

Bringing It All Together with Terraform

Now, let’s automate the deployment of the ArgoCD applications using Terraform. We’ll add the following to our IaC repo.

Applying ArgoCD Applications via Terraform

# In modules/argocd-apps/main.tf
resource "kubernetes_manifest" "application_sets" {
  for_each = fileset("${var.applications_path}", "*/overlays/${var.environment}/kustomization.yaml")
  
  manifest = {
    "apiVersion" = "argoproj.io/v1alpha1"
    "kind"       = "ApplicationSet"
    "metadata" = {
      "name"      = split("/", each.key)[0]
      "namespace" = "argocd"
    }
    "spec" = {
      "generators" = [{
        "git" = {
          "repoURL"      = var.devops_repo_url
          "revision"     = "HEAD"
          "directories"  = [{
            "path" = "applications/${split("/", each.key)[0]}/overlays/${var.environment}"
          }]
        }
      }]
      "template" = {
        "metadata" = {
          "name" = "{{path.basename}}"
        }
        "spec" = {
          "project" = "default"
          "source" = {
            "repoURL"        = var.devops_repo_url
            "targetRevision" = "HEAD"
            "path"           = "{{path}}"
          }
          "destination" = {
            "server"    = "https://kubernetes.default.svc"
            "namespace" = "{{path.basename}}"
          }
          "syncPolicy" = {
            "automated" = {
              "prune"    = true
              "selfHeal" = true
            }
          }
        }
      }
    }
  }
  
  depends_on = [var.argocd_module]
}

And update our main.tf:

module "argocd_apps" {
  source = "./modules/argocd-apps"
  
  applications_path = "./applications"
  environment       = var.environment
  devops_repo_url   = "https://github.com/your-org/devops-repo.git"
  argocd_module     = module.argocd
}

Continuous Delivery Workflow: Streamlining the GKE Deployment Process

With all these pieces in place, here’s how our continuous delivery workflow transforms the GKE deployment process for platform teams:

1. Infrastructure Changes:

• Make changes to the IaC repo
• Submit a pull request for team review (enforcing the “infrastructure as code” principle)
• After approval, run Terraform to apply infrastructure changes, including ArgoCD installation and configuration
• Time saved: Hours or days of manual configuration and potential human error eliminated

2. Application Configuration Changes:

• Make changes to the DevOps repo for ArgoCD application definitions or Kustomize configurations
• Submit a pull request for review
• After approval, commit and push changes to Git
• ArgoCD automatically detects and applies these changes without any manual intervention
• Time saved: Manual kubectl operations and environment-specific configurations eliminated

3. Application Code Changes:

• Update application code and Helm values in the application repo
• Submit a pull request for review
• After approval, commit and push changes to Git
• ArgoCD automatically detects and applies these changes based on the application definitions
• Time saved: Deployment processes that previously required DevOps engineer involvement now happen automatically

Key Workflow Benefits for Platform Teams

1. Self-service for Developers: Application teams can deploy to GKE without requiring deep Kubernetes expertise or direct cluster access
2. Reduced Context Switching: Platform engineers no longer need to juggle multiple tools and interfaces for deployments
3. Instant Rollbacks: If a deployment causes issues, reverting is as simple as reverting a Git commit
4. Environment Parity: The same process works identically across development, staging, and production environments
5. Visibility: The current state of all deployments is visible through both Git history and the ArgoCD dashboard

Conclusion: Transforming Platform Engineering with GitOps

In this guide, we’ve demonstrated how to manage ArgoCD using Terraform in a GCP environment and how to integrate Helm and Kustomize into a cohesive GitOps workflow. For platform engineering and DevOps teams working with GKE, this approach delivers transformative benefits:

• Infrastructure as Code: All components, including ArgoCD itself, are defined as code, eliminating configuration drift and manual processes
• Separation of Concerns: Different aspects of the system are maintained in separate repositories, allowing teams to work independently while maintaining system cohesion
• Environment Parity: Using Kustomize and Helm, we ensure consistent deployments across all environments, eliminating the “works in dev but not in prod” problem
• Automated Deployments: Changes to any part of the system are automatically detected and applied, reducing operational overhead by up to 80%
• Scalable Operations: This approach scales seamlessly from a handful of services to hundreds of microservices without requiring proportional growth in the platform team

According to the 2023 State of DevOps Report, organizations implementing GitOps practices like these see deployment frequencies increase by up to 200% and mean time to recovery (MTTR) decrease by up to 70%. For platform teams under pressure to deliver both speed and stability, these improvements can be game-changing.

Read my other tech blogs

Rahul Kumar Singh - Medium

Read Rohan’s tech blogs

Rohan Singh - Medium

Connect with us on LinkedIn: Rahul Kumar Singh and Rohan Singh

Clap and share this blog to show your support.