Simplify and Automate Deployments Using GitOps with IBM Multicloud Manager 3.1.2

Published in

IBM Cloud

10 min readFeb 15, 2019

GitOps and Multicloud may be two of the trendiest cloud native technologies (or buzzwords) for 2019. But did you know that you can combine the two concepts?

Now you can use IBM Multicloud Manager 3.1.2 with Argo CD as a technical preview to achieve declarative and automated deployment of applications to multiple Kubernetes clusters.

What is GitOps?

The term GitOps was coined by Weaveworks at 2017, although the concept started to emerge a bit earlier, borrowing and combining elements from continuous delivery, infrastructure as code, and Kubernetes’ natural affinity for declarative infrastructure. At its core, GitOps refers to a set of practices and tooling that puts Git at the center of the DevOps toolchain, and as the source of truth for what should be deployed on the cluster.

With GitOps, developers and operators use familiar Git workflows to define, review, approve, and audit changes to their infrastructure and applications, whereas automated tools take care of synchronizing the live state of their cluster with the desired state described in Git.

What is Argo CD?

Argo CD is a GitOps-style continuous delivery tool for Kubernetes, and is part of the larger Argo family of open source projects, primarily supported by Intuit. Despite being only recently launched, Argo CD is arguably one of the best open-source GitOps tools to date, due to its extensive feature-set, stability, robust and scalable implementation, effective documentation, and open and active community.

See some notable features for Argo CD; many of these are unique among GitOps tools:

Multiple Git repositories and cluster destinations
Multiple manifests template formats (Helm, Ksonnet, Kustomize), or plain YAML manifests
Git branch tracking or tag/commit pinning
Automated or manual synchronization
Pruning of deleted resources
Rollback/Roll-anywhere
Declarative management of its own configuration
Continuous monitoring of deployed applications
Audit trail and history for application events and API calls
Health assessment statuses on all components of the application
Web console, CLI, and gRPC/REST API
SSO Integration (OIDC, LDAP, SAML 2.0, and others)
Role-based access control
Webhook Integration (GitHub, GitLab, BitBucket)
PreSync, Sync, PostSync hooks for complex application rollouts (e.g. canary upgrades, blue/green)

Combining Argo CD with IBM Multicloud Manager

IBM Multicloud Manager is a recently launched product from IBM, offering three primary value-adds for organizations running a multicloud strategy:

Visibility across clouds
Automated application deployment across clouds
Compliance policies management and enforcement across clouds

IBM Multicloud Manager’s management components run on-top of an IBM Cloud Private Kubernetes cluster, dubbed as the hub-cluster, which manage and control a fleet of managed-clusters, running the IBM Multicloud Manager Klusterlet agent. Developers and operators can then interact with the hub-cluster to create template resources that describe applications and compliance policies to be deployed and enforced in the managed-clusters. While the Klusterlet agent works best with IBM Cloud Private clusters, it can also be installed in the following public cloud Kubernetes platforms:

IBM Multicloud Manager 3.1.2 bundles Argo CD as a technology preview feature. Argo CD’s GitOps capabilities complement IBM Multicloud Manager’s automated deployment and enforcement capabilities, by allowing users to manage their applications and compliance configuration resources in a centralized Git repository, and achieve end-to-end automated deployment and enforcement from their Git repository — down to the last of their managed clusters.

Let’s see how that works in practice.

Installing Argo CD to IBM Multicloud Manager

Prerequisites:

You need an accessible IBM Cloud Private cluster with IBM Multicloud Manager installed.
You also need a local installation of docker, kubectl, helm, and cloudctl CLIs. See IBM Multicloud Manager installation instructions prerequisites for further details.

Argo CD is distributed as part of the Passport Advantage (PPA) archive:

mcm-optional-components-x86–64–3.1.2.tgz.

If not already loaded, use the following steps to load its docker images and Helm chart to your hub-cluster:

First, login to your private Docker registry with the following command, where <cluster_ca_domain> is the certificate authority (CA) domain that was set for your hub-cluster. If you haven’t done so yet, you may need to configure authentication for your docker CLI:

docker login <cluster_ca_domain>:8500

Then, login to your hub-cluster with the following command, where <cluster_ip> is the IP address of your hub-cluster:

cloudctl login -a https://<cluster_ip>:8443 --skip-ssl-validation

Finally, load the PPA archive to your hub-cluster with the following command, where <cluster_ca_domain>, as above, is your hub-cluster‘s certificate authority domain, and <namespace> is the Kubernetes namespace in which you’d like to make the docker images available:

cloudctl catalog load-ppa-archive \
— -archive argocd-3.1.2-x86-64.tgz \
--registry <cluster_ca_domain>:8500/<namespace>

In a browser, login to the IBM Cloud Private management console of your hub-cluster, and follow these steps:

Click the Catalog button at the top-right corner.
Locate and click the ibm-argocd chart (available from the DevOps category).
Click the Configure button at the bottom-right corner.
Enter a Helm release name.
Select a namespace for your installation.
Note: This needs to be a namespace in which the previously pushed Argo CD docker images are accessible. For simplicity, the following example uses the same namespace that was used when the PPA archive was loaded.
Read and accept the license.
Enter an Ingress hostname for external access to the Argo CD console and CLI.
Note: Make sure to update your local DNS, or add an entry to /etc/hosts.
Optional: Expand the All parameters section and tweak any other installation parameter you want, such as enabling/disabling ingress, TLS settings, exposing as a NodePort service, etc.
Click the Local Install button at the bottom-right corner.

After a few short seconds, Argo CD will be up and running in your cluster.

At this point, you may want to read through the Helm chart installation notes. Click Menu > Workloads > Helm Releases > <helm_release_name> and scroll down to the bottom of the page to find quick start notes such as the Argo CD console URL, Argo CD CLI download link (optional), as well as instructions to locate and change the initial Argo CD login password.

Setting up a Git repository

To do GitOps, you clearly need a Git repository. Argo CD can work with any HTTP-based Git repository, but works best with GitHub (as well as GitHub Enterprise), GitLab, or BitBucket; so just choose your favorite one. I’ve set mine at https://github.com/zcahana/gitops-playground.

Now that you have your Git repo, your need to let Argo CD know how to access it.

Note #1: The next step is only required for repositories that need authentication. My repository at GitHub is publicly accessible and can be read by unauthenticated users, but since most real-world repositories do require authentication, I’ll proceed as if mine does, too.

Note #2: These instructions are for GitHub. Other Git services may require slightly different steps.

So, to begin with, head over to github.com, open your account settings, and click Developers settings > Personal access tokens > Generate new token. Since Argo CD only requires read access to the repository, choose the minimal OAuth scope for the token (“no scope” is usually sufficient) and hit Generate token.

With the token in hand, head over to Argo CD’s console, click the Settings button at the left pane, and then Repositories > Connect Repo. Fill in the repository’s URL (HTTPS clone URL) and credentials, and click Connect. Argo CD will soon indicate whether the connection to the repository is successful.

Alternatively, you can use Argo CD CLI to configure the repository. Run the following command:

argocd repo add https://github.com/zcahana/gitops-playground.git \
--username zcahana \
--password <access token>

Creating an Argo CD Application

You’re now ready to create your first multicloud application: a shiny new NGNIX server.

With IBM Multicloud Manager, you need to create several resources on the hub-cluster, to describe the application, its components, and its desired deployment target(s) (i.e., the managed-clusters to be installed on). The available resources are:

Application (sig-apps’)
ApplicationRelationship
Deployable
DeployableOverride
PlacementPolicy
PlacementBinding

I’ve checked-in such example manifests to https://github.com/zcahana/gitops-playground/tree/master/apps/mcm-nginx. You can use these as-is, or create your own and check-in to your repository of choice.

Next, you need to create an Argo CD Application (not to be confused with the previously mentioned sig-app’s Application). In Argo CD, an Application is a logical grouping of Kubernetes resources that share the same source (Git repository and revision, directory path), the same destination (target cluster and namespace), and are synced together.

To define the application, head over again to Argo CD’s console, click the Apps button at the left pane, then +> New Application. Then, input your Git repository address, and click Next. Here, Argo CD lists all of the Helm/Ksonnet/Kustomize source directories found on the repository, and you can choose one of them to create an application from. For plain YAML manifests, as I’ve used in the example, click the Create app from directory button to the right. In the next screen, fill in the application details:

Repository URL: This is the Git URL in which the application sources reside.
Revision: This is the Git revision that is tracked for this application. By default this is set to HEAD (which translates to the default branch for the repository), but can be set to a name of a branch, or even pinned to a specific tag to commit hash. See Tracking Strategies for more information.
Path: The directory path within the Git repository, in which the application sources reside.
Project: The Argo CD Project to which the application belongs. For simplicity, you can use the preconfigured default project. See the “What’s Next” section below for some more details.
Application Name: The name of the Argo CD application.
Cluster URL: Argo CD can be configured to deploy applications in remote Kubernetes clusters, but since you’re using IBM Multicloud Manager to intelligently distribute and manage resources across multiple clusters, just pick https://kubernetes.default.svc which refers to the local cluster (the hub-cluster).
Namespace: The default namespace for the application. This is used when there’s no namespace defined in a manifest, similarly to kubectl apply --namespace or helm create --namespace.

Once done, click Create and you’re back at the applications view. There, click the application’s tile to get into the application details page. Here, you’ll find a topology view of the application and its resources, its synchronization and health status, as well as various options to tweak the application (e.g., enable auto-sync, modify tracked revision, etc). Note that by default, auto-sync is disabled when creating applications via the console, so you’ll see that your application is in OutOfSync status — meaning there’s a difference between the desired application state as defined in its manifests in Git, and the actual state of the live objects in the cluster.

To synchronize the application, you can either manually sync it (⋮ > Sync) or enable auto-sync (⋮ > Details > Summary > Enable auto-sync). Similarly, you can also enable automated pruning, in which the synchronization process also includes deleting resources that are no longer in Git.

As usual, you can save yourself a lot of clicks by creating the application with the Argo CD CLI. Note that in the following command, auto-sync and auto-prune are enabled right away during application’s creation:

argocd app create nginx \
--repo https://github.com/zcahana/gitops-playground.git \
--path apps/mcm-nginx \
--dest-server https://kubernetes.default.svc \
--dest-namespace default \
--project default \
--sync-policy automated \
--auto-prune

After Argo CD synchronizes the application and creates its resources in the cluster, IBM Multicluster Manager controllers kick in and do the work — in this case, constrollers install an NGINX Helm chart in one of the managed-clusters, according to the PlacementPolicy. Here’s how it looks from the IBM Multicloud Manager console’s Applications page:

What’s next?

Argo CD packs a wealth of other useful features, and specifically for real-world usage. You can read more about them in Argo CD documentation:

Single Sign On: Can be used to integrate Argo CD with the IAM service of your choice (and particularly, with IBM Cloud Private IAM services).
Role-based access control: Allows to restrict users’ access to Argo CD resources (Applications, Projects, Repositories, etc).
Projects: In Argo CD, a Project is a logical grouping of Applications, and can be used for restricting access of an application in multi tenant environments. Projects can restrict what may be deployed (the Git source repositories), where applications may be deployed (the target clusters and namespaces), and what type of resources may be deployed (e.g. Deployments, Secrets, etc).
Webhooks: By default, Argo CD polls Git repositories every 3 minutes, to retrieve updated application manifests. This can be significantly improved by configuring a webhook, so that Argo CD immediately retrieves updated manifests whenever they’re pushed/merged to the Git repository.
Declarative Setup: Argo CD’s own configuration is expressed as Kubernetes resources (using the Application and AppProject CRDs, as well as ConfigMaps and Secrets) , and too can be managed declaratively using Git.
Sealed Secrets: An open source project from Bitnami Labs, which allows to securely store encrypted secrets in Git repositories.