GitOps Platform Powered by Argo CD

Hi, my name is Sergey, I am the head of the DevOps unit at Lonto.

We periodically partner with interesting guys and organize events together. The examples of such events are meetings of entrepreneurs, online flat parties, and product management days. This series of three articles is a text version of the speeches from the CTO Day event. The report has two speakers:

• Sergei Malenko, Head of DevOps unit at Lonto
• Sergey Bondarev, Architect at Southbridge

The report was devoted to the history of app deployment, the main models used for it, and their comparison. We discuss the pull model in detail and show how to use “advanced” tools to manage the infrastructure of large projects and allow developers to independently order elements in the infrastructure for the needs of their apps.

All in all, we have prepared three articles based on the report. They will follow this order:

The history and how things evolved
To understand the context, we will check out how the approach to iterative app deployment changed over time.

What is ArgoCD and why do we need it, with use cases ←you are here 🚩
We’ll talk about a relatively new turn in the app deployment methods and see what questions can be solved with this tool.

How to manage infrastructure in GitOps using Crossplane
A new approach to IaC and how it can be combined with Argo CD

GitOps is one of the implementations of the pull model, in which Git is the repository of all configurations. Git is the source of truth, it is the only point through which the changes to the infrastructure are done. All pull-model changes are made by a special agent, which then maintains the set state. That is, if you make changes to the infrastructure manually, the agent will notice a discrepancy with what’s in Git and return everything to the correct state, identical to the source of the truth.

The Argo CD is one of the most popular GitOps tools. It lives inside Kubernetes and deploys entities there. Argo CD provides a handy RBAC, that is rights and access management. In the interface you can track your actions, manage apps, and forcibly synchronize them. The Argo CD is a part of the CNCF, which gives it a lot of credibility.

Content:

• Argo CD basics
• Operation scheme
• Interface
• Updating apps
• Pros and cons of Argo CD
• Argo CD API
• ApplicationSet
• Examples of using ApplicationSet
• How does ApplicationSet look like
• The bottom line: pros and cons of the GitOps approach

Argo CD basics

Argo CD can deploy into a cluster:

• Common manifests
• Kustomize
• Jsonnet
• Helm charts

These manifests can be taken from the Git or Helm repository. Argo CD deploys manifests to Kubernetes clusters only, but not only to the cluster in which it resides itself. It is able to manage a large number of clusters simultaneously.

The main entities that this tool operates with are:

• cluster
• repository
• application

Application is the root entity in the Argo CD. It specifies what and where to deploy. The “application” entity is bound to a single “project” object, which is needed to distinguish access rights. There could be many “projects” inside Argo CD. Let’s say these are folders for demarcation of access rights and a convenient hierarchy of apps.

How the application manifesto looks like:

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
 name: app
 namespace: argocd
spec:
 # what
 source:
   path: app
   repoURL: https://gitlab.ktsstudio.ru/argocd/app.git
   targetRevision: HEAD
 # where
 destination:
   namespace: app
   server: https://kubernetes.default.svc
 # project
 project: default

Everything Argo CD operates on is stored in the cluster itself: all entities are separate “Custom resources” with their own apiVersion and “kind.”

The figure above shows an example of an “application” that deploys an app. In “source,” we specify from which repository and which folder to take the chart. In “destination” we specify the cluster and “namespace” into which we will deploy. And in the “project” we define which project our app will belong to.

Operation scheme

You can find a description of how the Argo CD works on the official website. We’ve got an API here that the user interacts with either through the web interface or through the client. There is a repository service that works with repositories and keeps a cache. And the heart of the solution is the Application Controller, which compares the current state of the infrastructure with what is described in the repositories. If changes occur, the app goes into sync status.

Interface

The Argo CD has a user-friendly, visual interface. We see the deployment process and everything that is included into the app. Everything is represented as a graph. For example, it displays the deployments, the replica sets created from them, and finally the pods which have been deployed by the replica sets.

Updating apps

You can update apps in the Argo CD both manually and automatically.

Manually

Manually

In the first case, the developer pushes the project into Git. An image is assembled. Then it is sent to the registry. Next, you need to manually change the tag of this image in values.yaml in the Git repository with charts. The Argo CD detects this and, depending on where the changes were made, applies them to the right environment. Manual deployment raises security concerns: you have to give the developer access to the repository with the charts where the values files are stored.

Automatically

Automatically

Naturally, we are more interested in this option. The Argo CD has a separately installed Image Updater component which constantly monitors the Container Registry. As soon as the new version arrives there, it changes the image version in the chart itself and applies the changes to the clusters.

Still, we are not bound to take the latest version every time. We can also use more complex strategies, such as working with regexes or using semantic versions.

Pros and cons of Argo CD

Pros:

1. Constant synchronization. If Helm fails and hangs up in our case, Argo CD will constantly try to apply changes to the infrastructure.
2. User-friendly interface. Very visual representation of entities deployed in Kubernetes.
3. Viewing the logs. We open the pod, go to the description, and see the Logs tab. It will display all the events that took place. You can see the logs of all running and failed pods. As long as there are few logs, such as 50 lines, everything works quickly. But if their number increases, it is worth using the web interface of a separate log collection system for viewing.

Cons:

1. Adding apps manually. For example, we bought a new cluster and want to add something to it. We will have to manually add a new “application” to the repository and specify where to and where from we are adding projects. Because of the difficulty of adding new apps, there is another disadvantage.
2. Impossibility to create the environment dynamically. It is not possible to track the moment when a developer pushes something into a branch. We will still have to go to another repository, create an “application” there, and only then will our app appear. But there are two solutions to this problem.

In fact, these disadvantages are solved in Argo CD. We will consider this further by comparing two approaches to solving them.

Argo CD API and push model

If we’ve used Helm, we remove the Helm Upgrade command and do kubectl create app.yaml, kubectl upgrade app.yaml instead. Accordingly, the Argo CD will start to track all changes. When we make a new dynamic environment, we create these custom resources through the Argo CD API or directly post a YAML with the manifest to the Kubernetes cluster. The result is an automatic dynamic configuration of the Argo CD.

Let’s consider the second option to solve the problem: using a powerful ApplicationSet tool.

ApplicationSet: staying within the pull model

Argo CD has ApplicationSet, its own tool for dynamically creating “application” within a pool model.

It allows you to create apps automatically instead of manually. For this you need two components:

1.Template, that is, a description of what to generate.
2.Rules

Rules can be different: a list of values, clusters added to Argo CD, or a folder in Git. When we add a new chart to the Git folder, it automatically creates an “application.”

Important point: With the Argo CD, you can work with repositories of SCM, such as GitLab, GitHub, and Bitbucket. You can also monitor the appearance of new branches in them. Rules can be combined, that is you can do multiplication. For example, you can multiply clusters by a Git folder with charts. That way, a set of charts from this folder will be installed in each cluster.

Examples of using ApplicationSet

Cluster setup

Let’s say we have a production and a stage cluster and we want to add a dev cluster.

Usually after creating a cluster, we need to put infrastructure apps there. They are necessary for the correct work of developer apps. For example, Ingress controller, cert-manager, and monitoring stack to keep track of what is going on inside the cluster.

How does the ApplicationSet help here? We configure it to track a folder with charts and deploy those charts to all the clusters. When we add a new cluster to the Argo CD, it immediately starts deploying all these charts into it. The Ingress controller, cert-manager, and VictoriaMetrics will appear as an example of a monitoring stack. This way you can ensure that all clusters have the necessary set of infrastructure tools. There is no manual action here: except for adding a cluster, everything else will be done by Argo CD. If we want to add a new item to all the clusters, we just add it to the infrastructure charts folder and everything will be deployed automatically.

Dynamic environments

Example: There is a team of developers who work on two projects. They use a common methodology and develop features in separate branches with names like feature-<task number>. We create an ApplicationSet, configure it with the Gitlab group that contains these projects, and specify that you want to track the feature-* type branches. As a result, Argo CD will generate “application” entities for each individual developer branch.

Full automation (extreme option)

This is an example for those who are not afraid to experiment. Here we create several ApplicationSet objects. Then the developer simply creates a project in the desired group. Argo CD and its ApplicationSets see that there is a new project in the group. They immediately deploy a production, stage, and dynamic environments for it when the developer creates branches in these repositories.

How does ApplicationSet look like

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata: ...
spec:
 generators:
   - scmProvider:
       # ...
       filters:
         - branchMatch: ^feature
           repositoryMatch: ^backend
       gitlab:
         allBranches: true
         api: "https://gitlab.team.ktsstudio.ru"
         group: applicationset
         ...

In the “generators” section, we specify the rules for generating “applications.” In this case, we specify GitLab, keep track of all repositories that start with “backend,” and deploy only branches that start with “feature.”

In the “template” section, we specify the template used to create the “application.” Note that you can use various ready-made variables from Argo CD inside (e.g. “repository” and “branch”).

template:
 metadata:
   name: >-
     {{ printf "{{ repository }}-{{ branch }}" }}
 spec:
   destination:
     namespace: >-
       {{ printf "{{ repository }}-{{ branch }}" }}
     name: dev-cluster
   project: my-service
   source:
     # ...path, repoURL, targetRevision
     helm:
       parameters:
         - name: "ingressHost"
           value: >-
             {{ printf "{{ branch }}.example.com" }}

Generation example:

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
 name: backend-auth-feature-1
 namespace: argocd
spec:
 destination:
   namespace: backend-auth-feature-1
   name: dev-cluster
 project: my-service
 source:
   # path, repoURL, targetRevision...
   helm:
     parameters:
       - name: "ingressHost"
         value: feature-1.example.com

The bottom line: pros and cons of the GitOps approach

The classic push model uses CD Jobs everywhere, and you can only introduce changes through them. Hence the problem. For example, if you want to change the way a certificate is obtained, you have to go to each repository separately, make changes, run the pipelines, and only then everything is synchronized.

With the Argo CD you can remove this step. Argo only needs GitLab and the Docker Registry, by which it will generate “application” entities and update them. That is, with this tool we solve all synchronization problems.

Pros and cons of the GitOps approach

Pros:

1. Control over actions on the infrastructure. We can see all actions and control access to the infrastructure through GitLab.
2. Constant synchronization.If someone makes changes manually, Argo CD will constantly try to roll them back. That is, we get protection from unnecessary edits. At the same time you can disable synchronization in the Argo CD. For example, if you need to test something.
3. Easier management of a big infrastructure For example, Argo CD is very helpful when you need to add a new Ingress controller to each cluster of a sprawling infrastructure or to adjust the monitoring stack settings.

Cons:

1. More complicated implementation. If you have one project and little infrastructure, it’s easier to use the usual pipelines and the push model.

The next part of the report will be about managing the infrastructure itself using the new Crossplane tool. It will allow us to create any objects in the cloud and synchronize their state using a Kubernetes cluster only.