Mastering ArgoCD Management: Advanced GitOps Techniques for Flexibility and Scale

Are you ready to elevate your ArgoCD deployment to unprecedented heights?

In this article i’ll provide implementation details of ArgoCD deployment that allows flexibility, scalability and easy maintenance.

Managing ArgoCD With ArgoCD

The first thing we will do is having the ability to configure ArgoCD installations in a GitOps fashion.

Picture managing ArgoCD with ArgoCD itself.

To bring this concept to life, we’ll initiate the installation of a “dumb” ArgoCD instance using Terraform.

This instance will house a “bootstrap application,” acting as the gateway to deploying the actual ArgoCD instance of our desire. Our Terraform script is designed for simplicity; execute it once, and it gracefully fades into the background. Let’s dive into some pseudo-code to demonstrate how this ArgoCD instance inception occurs:

resource "helm_release" "argocd" {
  depends_on = [
    module.gke_auth
  ]
  name             = "argocd"
  namespace        = "argocd"
  create_namespace = true
  repository       = "https://argoproj.github.io/argo-helm"
  chart            = "argo-cd"

  values = [
    file("${path.module}/custom-values.yaml")
  ]
}

resource "helm_release" "argocd-apps" {
  depends_on = [
    helm_release.argocd
  ]
  name             = "argocd-apps"
  namespace        = "argocd"
  create_namespace = true
  repository       = "https://argoproj.github.io/argo-helm"
  chart            = "argocd-apps"

  set {
    name = "applications[0].name"
    value = var.argocd_bootstrap_app_name
  }

  set {
    name = "applications[0].source.repoURL"
    value = var.components_repo_url
  }

  set {
    name = "applications[0].source.targetRevision"
    value = var.components_repo_url_target_revision
  }

  set {
    name = "applications[0].source.path"
    value = var.argo_bootstrap_app_path
  }
}

This Terraform code executes the installation of two critical Helm charts: one for the “dumb” ArgoCD instance and another for the bootstrap ArgoCD application. The bootstrap app, crafted with the “App of Apps” pattern, resides in a designated path within a connected Git repository. Within this Helm Chart lies the installation instructions for various ArgoCD “Application CRDs,” among which a pivotal one orchestrates the deployment of the true ArgoCD instance as per our vision.

What’s extraordinary is the GitOps integration at play. Every facet is governed by the Git repository. Need a tweak in the “real” ArgoCD server? A simple commit and push is all it takes. This foundation merely sets the stage for the remarkable capabilities we are about to unveil.

ApplicationSet Helm Chart

Central to our setup is the ApplicationSet Helm Chart. The ApplicationSet is an ArgoCD resource type that orchestrates the generation of multiple ArgoCD applications. This generation is orchestrated through diverse logics and use cases, empowering an array of possibilities.

Envision the architecture: the “dumb” ArgoCD instance boasts a “bootstrap application” responsible for spawning two new applications. The first deploys the “Real ArgoCD” instance via the official ArgoCD Helm chart. The second installs an ApplicationSets-customized Helm Chart, introducing “ApplicationSet” CRDs to the namespace of the true ArgoCD instance, exclusively managed by it.

Picture a custom Helm chart for ApplicationSets, iterating through applications destined for your target Kubernetes clusters. Every application and cluster can be tailored via the Chart’s values.yaml:

{{- range .Values.applicationSetsList }}
apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: {{ .name }}-applicationset
  namespace: {{ $.Values.argoName }}
spec:
  goTemplate: true
  generators:
    - clusters:
        selector:
          matchLabels:
            clusterType: workload
            managedBy: {{ $.Values.argoName }}
            envType: prod
        values:
          defaultTargetRevision: {{ $.Values.envs.prod.targetRevision }}
  # optionally you can add here more clusters types and logics
  template:
    metadata:
      name: "{{`{{.name}}`}}-{{ .name }}"
      finalizers:
        - resources-finalizer.argocd.argoproj.io
    spec:
      source:
        repoURL: {{ .repoURL | default $.Values.source.repoURL }}
        targetRevision: "{{`{{ default .values.defaultTargetRevision .metadata.annotations.targetRevision }}`}}"
        path: "{{ $.Values.source.chartsPath }}/{{ .name }}"
        helm:
          valueFiles:
          - values.yaml
          {{- if .valuesOverride }}
          - "path to values.yaml with higher importance"
          {{- end }}
      syncPolicy:
        syncOptions:
        - ServerSideApply=true
        - CreateNamespace=true
      destination:
        server: "{{`{{.server}}`}}"
        namespace: default
      project: default
---
{{- end }}

And the values.yaml:

# The name of the argocd instance that responsible for the applicationsets
# can be overriden in another values.yaml thus creating many "real argocd instances"
argoName: "real argocd name"

source:
  repoURL: "path to the repository"
  chartsPath: "path to helm charts"

envs:
  prod:
    targetRevision: "the target revision of the applications"

applicationSetsList:
  - name: loki-stack

  - name: grafana

  - name: ingress-nginx
    valuesOverride: true

  - name: prometheus

All the point of this Helm Chart is that you can customize it endlessly according to your own needs. Feel free to play with it.

Target Revision Flexibility

Enter a game-changing capability: overriding cluster Target Revisions on-the-fly without Git commits. The secret lies in a specific section of the ApplicationSet chart:

# Extracting targetRevision flexibly from the metadata annotations of the cluster secret
targetRevision: "{{`{{ default .values.defaultTargetRevision .metadata.annotations.targetRevision }}`}}"

This line means that be default we are injecting the “TargetRevision” of the Application to be the value from the helm chart.

envs:
  prod:
    targetRevision: "the target revision of the applications"

But if it happens that we want to change the TargetRevision of all the Applications of the target cluster, due to some HotFix or testing a feature. All we need to do is editing the target cluster annotations from the ArgoCD UI or from the k8s API. All clusters managed by ArgoCD are represented as a k8s Secret object inside the cluster, what we need to do is adding the following annotation to the secret of the wanted cluster:

annotations:
  targetRevision: "my-new-revision"

What will happen behind the scenes is that the following line:

.metadata.annotations.targetRevision

will extract the annotation from the k8s Secret and will immediately apply it to all the Applications of this cluster. When we finished just remove the annotation and the cluster will automatically go back to the default target revision that we defined in the helm chart.

Scaling up the number of ArgoCD instances, and flexibility for changing Which Argo manage which k8s cluster.

As organizations grows, the need for scalability becomes evident. Here, the ArgoCD managed by ArgoCD concept shines, positioning ArgoCD as a versatile Kubernetes application that can span diverse clusters with distinct configurations — all through GitOps.

Consider this scenario: a single ArgoCD instance buckles under the weight of applications and clusters. Our architecture effortlessly accommodates the deployment of additional ArgoCD instances. The mechanism is simple: the “Dumb” ArgoCD instance’s bootstrap application orchestrates the installation of “Real” ArgoCD instances. For achieving this feature we can loop over a list of desired “Real” instances.

Example:

{{- range .Values.argoCDInstances }}
---
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: {{ . }}
  finalizers:
    - resources-finalizer.argocd.argoproj.io
  namespace: Namespace of dumb argocd
spec:
  source:
    repoURL: {{ $.Values.source.repoURL }}
    targetRevision: {{ $.Values.source.targetRevision }}
    # Path to official ArgoCD Helm Chart
    path: {{ $.Values.source.chartsPath }}/argocd
    helm:
      valueFiles:
      - values.yaml
      # Path to values.yaml that will customize the specific ArgoCD instance
      - "/configuration/argocd-definitions/{{ . }}/values.yaml"
  syncPolicy:
    syncOptions:
    - ServerSideApply=true
    - CreateNamespace=true
    automated:
      prune: true
  destination:
    server: https://kubernetes.default.svc
    # Install the ArgoCD instance in it's own namespace
    namespace: {{ . }}
  project: default
{{- end }}

Pretty cool no?

But how all this ArgoCD instances will know which applications/clusters to manage?

The secret is in the following code section inside the ApplicationSet Helm Chart:

generators:
    - clusters:
        selector:
          matchLabels:
            clusterType: workload
            # When we adding a cluster into ArgoCD we will also 
            # add a "managedBy" label in order to determine which ArgoCD
            # instance manage this cluster
            managedBy: {{ $.Values.argoName }}
            envType: prod
        values:
          defaultTargetRevision: {{ $.Values.envs.prod.targetRevision }}

Facilitating this orchestration are the clusters’ generators within the ApplicationSet Helm Chart, dictating which ArgoCD instance governs which clusters. The concept of “managedBy” labels brings this orchestration to fruition, creating a harmony between ArgoCD instances and their designated clusters.

Summary

In this transformative exploration, we’ve uncovered methods to wield ArgoCD’s potential at an entirely new level. Managing ArgoCD with ArgoCD itself catapults us into the realm of GitOps for configuration control. The ApplicationSet Helm Chart empowers us with endless strategies for seamless deployment, all neatly synced through GitOps. Flexibility in Target Revision injects agility without compromising GitOps principles. And finally, scaling ArgoCD instances propels us into a future of unmatched scalability and multi-tenancy.