Deploy Instana agent on OpenShift with GitOps

Co-Authors: Xiao Lu, Tian Xing, Mo Ying

About GitOps

GitOps is an operational model that employs DevOps best practices and Infrastructure as Code (IaC) to perform operations on infrastructure and modern cloud native applications. It puts the descriptive code of a cloud infrastructure or application into a git repository for process and version control, and then applies the changes to infrastructure by automation.

GitOps brings many benefits, including unified source management, easy rollback and fast recovery, and enhanced security. It uses a git repository as a single source to manage infrastructure configuration change. All changes on infrastructure are version controlled and reviewed and therefore easy to roll back on.

About Instana

Instana is a fully-automated Application Performance Management solution designed mainly for the challenges of managing microservices and cloud-native applications. In June 2022, IBM Instana® was named a Leader in the 2022 Gartner® Magic Quadrant™ for APM and Observability. More and more companies are adopting Instana as their hybrid cloud APM and Observability solution.

Instana and GitOps

This article proposes a method to deploy an Instana agent on OpenShift with GitOps.

There are several tools developed for GitOps practices. Since Argo CD is integrated into Red Hat OpenShift GitOps, you can use Argo CD as the GitOps tool. Although for the purposes of this article Argo CD is installed by Red Hat OpenShift operator, it can also be installed in Kubernetes. The GitOps solution to deploy an Instana agent can also be applied to Kubernetes.

The following diagram displays a high-level architecture when applying GitOps to deploy Instana agents. The developer changes the application source code and merges the changes after reviewing the pull request. Then the CI pipeline detects the changes and triggers the build into an Artifacts repository. The SRE makes the changes to the environment configuration in the Declarative Config Repo and merges the changes after the review process. The GitOps agent then detects the changes in the environment configuration repo, pulls the latest configuration, and applies those changes to the environment.

In this case, you can leverage the Instana public artifactory (https://agents.instana.io/helm) to pull the image. If you’d like to use your own private registry, which may already contains the Instana agent images, you need to set up the registry server CA trust in your cluster.

Prerequisite:

You have installed and set up at least three OCP clusters, one of which will be used for the Argo CD instance, and the others will install Instana agents.

Installing Red Hat OpenShift GitOps Operator

First, log on to the OCP web console of your cluster, and install Red Hat OpenShift GitOps Operator from the OperatorHub.

Then click the Cluster Argo CD link from the menu. The login page of the Argo CD UI will be displayed. You can refer to these instructions to find the password of your Argo CD admin account.

You can log in to the Argo CD UI now. The main dashboard of Argo CD UI is shown in the applications view.

Registering repositories and clusters to Argo CD instance

If you are using a GitHub Enterprise repository or a private non-GitHub-Enterprise repository, you need to register the repository to your Argo CD instance. Click the Settings button in the side bar of the Argo CD UI and add the repository where your source code is located.

Then click the Doc button in the side bar of the Argo CD UI and download the CLI tool.

Now you can add OCP clusters to the Argo CD instance through CLI.

export USER=kubeadmin
export PASSWORD=<cluster_password>
export CLUSTER_LONG_NAME=https://api.apm2.com:6443
export CLUSTER_SHORT_NAME=apm2
export ARGOCD_SERVER=openshift-gitops-server-openshift-gitops.apps.cluster1.com
export ARGOCD_USER=admin
export ARGOCD_PASSWORD=<argocd_password>
export InstanaNamespace=instana-agent
oc login -u $USER -p $PASSWORD $CLUSTER_LONG_NAME — insecure-skip-tls-verify=true
CLUSTER_CONTEXT_NAME=$(kubectl config view -o jsonpath=’{.current-context}’)
echo $CLUSTER_CONTEXT_NAME
oc config rename-context $CLUSTER_CONTEXT_NAME $CLUSTER_SHORT_NAME — insecure-skip-tls-verify=true
argocd login $ARGOCD_SERVER — username $ARGOCD_USER — password $ARGOCD_PASSWORD
argocd cluster add $CLUSTER_SHORT_NAME -y
argocd cluster list|grep $CLUSTER_SHORT_NAME
oc new-project $InstanaNamespace
oc logout

When the commands finish, you can see the cluster named apm2 listed in the UI now.

Defining the helm chart to install the Instana agent and pushing to the Git Repo

Now it’s time to define the helm chart to install the Instana agent on your Openshift cluster. The Chart.yaml file defines this helm chart as a proxy which points to the official Instana agent helm chart (https://github.com/instana/helm-charts). The values.yaml file is based on the official Instana agent helm chart values.yaml file (https://github.com/instana/helm-charts/blob/main/instana-agent/values.yaml). Besides, you can also define dedicated values_apm2.yaml file for your cluster, which will overwrite the default values set in values.yaml.

Chart.yaml:

apiVersion: v2
appVersion: 1.213.0
description: Instana Agent for Kubernetes
name: instana-agent
version: 1.2.32
type: application
dependencies:
- name: instana-agent
  version: 1.2.32
  repository: https://agents.instana.io/helm

values_apm2.yaml:

instana-agent:
  agent:
    endpointHost: ingress-saas.example.com
    endpointPort: 443
  cluster:
    name: apm2
  zone:
    name: test

When the helm chart and values files are ready, you can commit the changes and push them to your git repository.

Defining ApplicationSet resource

Now you can go back to your Argo CD instance. As you may know, Argo CD provides ApplicationSet controller to manage deployments of a large number of applications, repositories, or clusters, all from a single Kubernetes resource.

If you have multiple clusters to deploy the Instana agent, you can define a ApplicationSet resource with Cluster generator as below.

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: appset-helm
spec:
  generators:
  - matrix:
      generators:
        - git:
            repoURL: https://github.ibm.com/xiaolu/gitops-instana-agent.git
            revision: HEAD
            directories:
              - path: helm-charts/instana-agent
        - clusters:
            selector:
              matchLabels:
                argocd.argoproj.io/secret-type: cluster
  template:
    metadata:
      name: ‘{{path.basename}}-{{name}}’
    spec:
      project: default
      source:
        repoURL: https://github.ibm.com/xiaolu/gitops-instana-agent.git
        targetRevision: HEAD
        path: ‘{{path}}’
        helm:
          valueFiles:
          - ‘values_{{name}}.yaml’
      destination:
        server: ‘{{server}}’
        namespace: ‘{{path.basename}}’

The ApplicationSet controller automatically generates Argo CD Applications for each cluster. You’ll see new applications displayed with initial Missing and OutOfSync states in the Argo CD Applications dashboard.

Synchronizing Argo CD Applications

Click the SYNC button for the application “instana-agent-apm2”, then a pop-up window is shown with synchronization options. Argo CD automatically detects differences between the desired manifests in Git and the live state in the cluster, and marks those out-of-sync resources with yellow icon. Make sure all OutOfSync resources are selected then click the SYNCHRONIZE button.

After a while, all Kubernetes resources are created and the application is shown as Healthy and Synced. The Instana agent has been deployed successfully.

You can perform the same SYNC actions for other ArgoCD applications. Then Instana agents can be deployed to all other clusters.

Changing Instana agent configuration

If you would like to change the agent configuration for all your clusters–such as requested memory allocation for agent pods–you can change the basic values.yaml file in the git repository directly.

pod:
  requests:
    memory: 768Mi
    cpu: 0.5
  limits:
    memory: 1024Mi
    cpu: 1.5

Then this git change will be detected by Argo CD and be applied to all your clusters according to the synchronization policy.

Conclusion

GitOps provides benefits of fast deployment, quick detection and apply of configuration changes, easy rollback and recovery for infrastructure and cloud native applications. You can leverage GitOps to carry out large-scaled Instana agent deployment and management, which helps improve operation tasks in a more efficient way.