Making Your Way into GitOps

DevOps and agile methodologies have become industry standard in the software release market. We’ve all heard about automating software release process tasks aiming to reduce time to market and allowing integration for developers and operations teams helping the business removing the well-known silos of the old corporate way of management.

The fundamental pillar of this new agile method is: “Having a unique (single) and well-managed source of truth”; so, what does this mean?

A single source of truth (SSOT) is the practice of structuring information so that everyone uses the same data. In version control, a single source of truth means storing all code, configuration, and other digital assets in a way that everyone can access them in a common location.

We all know Git and how it has become a fundamental tool in the modern way of developing software. With these things in mind, I believe that GitOps is the expected evolution of operations, allowing businesses to manage their infrastructure the same and familiar way as source code.

This is only possible thanks to modern infrastructure platforms that use a declarative approach for defining infrastructure components. Container orchestrators such as Kubernetes are the best example.

The main purpose of this article is to provide a step-by-step guide that will help you and your business ease your way into implementing a full GitOps workflow in your software release chain.

Requisites:

1. Support your infrastructure in a platform that allows you to define your infrastructure in a declarative way. (E.g. Kubernetes)
2. Choose your GitOps tools (E.g. GitHub + ArgoCD)
3. Know your maturity level.

Step-by-Step:

1. Create your GitOps source of truth:

Three options:

a. Unique Git repository.

b. Use an app or microservice current source code repository.

c. Create a new repo per application just to define the infrastructure for the service.

From my experience option, a is the best approach if the number of applications is not huge.

2. Choose the easiest and simple application or micro-service as the first one to implement the CD workflow using the GitOps approach.

3. Implement this in phases based on the environment (Dev, QA, Prod).

4. Move your way into your software stack in the same environment.

5. Close phase 1 implementing full GitOps on Dev environment.

6. Educate your operations and developers force into the new way of doing things. Close the gap ensuring no one can make changes on the deployments connecting directly to the infrastructure; all the updates or modifications to the currently deployed application have to go through the whole workflow.

7. Implement the same in QA or testing environments.

8. Implement this into production only when you’re 100% sure that the workflow on the other 2 environments is working for you.

Here’s a graphic representation of the proposed GitOps workflow to implement with the listed steps.

What you’ll gain with the GitOps approach

1. Manage the configuration for the infrastructure components as you currently do with your source code.
2. All the updates to the infrastructure components definitions will be traceable and revertible in an easy and straightforward way.
3. Better integration for your development and operations teams, reducing the friction and promoting collaboration between them. Every member of the team, regarding its role, will have visibility of the software release process from beginning to end.
4. Pack the application or microservice with its infrastructure components allowing your organization to be more agile when releasing new software.
5. Increase the security of your infrastructure by applying the less privilege principle. Your CI/CD tools don’t need credentials for the deployment step in the pipeline; deploying a new version consist in modifying the corresponding component on the Git repo that hosts the definition for the infrastructure components, the GitOps tool will do the rest making sure the running components are synced with the definition specified in the Git repo.
6. Handle all your access management from the Git tool.
7. Increase in the productivity for your teams allowing them to release updates and new software frequently leveraging the more complex part of the process (deploying) to specialized tools and automation.
8. Developers don’t need a deep infrastructure understanding in order to define the required configurations to deploy new code or new applications.
9. Increase stability and reliability of the infrastructure. No one will be able to make modifications or updates on the infrastructure without updating the corresponding Git repository and having the required authorizations and reviews like we do for code.
10. Standard deployment definition for infrastructure components. A templating approach (using tools like helm or kustomize) allows you to replicate deployments from application to application using the same configuration as code definitions.

Disadvantages of the GitOps approach and how to cope with them

1. GitOps only governs a subset of the activities executed in the release of the software process and there’s a thin line between CI and CD activities and sometimes it is hard to draw it properly. You’ll need to define a automate properly your CI/CD activities and leave GitOps as the CD strategy.
2. There are no standards defined to deploy code in several environments. Organizations are free to choose the approach and sometimes it does not suit its needs and may complicate things. To cope with this, choose the approach that works for your organization depending on the number of environments you have, basic approach consists of promoting code between branches and having each branch deploy to an specific environment or set of environments.
3. There are no standards defined to handle secrets and delicate data. My recommendation for this will always be to leave secrets out of the process and handle them as a separate unit using a vault tool.
4. GitOps may interfere with autoscaling and dynamic resources creation. E.g. k8s deployment definition declares the creation of “1” replica interfering with the HPA min value of 2 for that deployment. The “self-heal” option can be disabled on ArgoCD to allow the deployments to scale freely depending on the HPA specification.
5. It might be hard to implement GitOps at big scale. If you decide to have a unique repo to handle your infrastructure definitions, it will grow without control with the inclusion of new applications; if you go with the one Git repository per application approach, the number of Git repositories will grow without control. To fix this, you can divide your applications depending on business logic and have one Git repo per business unit, this will allow the services to be divided and categorized according to it’s functions and reduce the amount of repositories and their size.

With this framework and the pros and cons listed, I hope you and your organization have a better understanding of the implications of going all the way with GitOps. Is my opinion that the benefits of the practice are far greater than the disadvantages.