Since ArgoCD 2.8 was released we the Users/Operators, have been granted the ability to create our own Generator via the Plugin-Generator, which I explain here more deeply, but to make a long story short this new generator enables us to build any application that can receive any payload and in return post a JSON body that can be used in the ApplicationSet template YAML.
So now after I have created a simple web application based on NodeJS that can take YAML files mounted via config-map with the help of HELM, I thought to myself, The argoCD PullReqeust Generator is a great tool to create Disposable deployment to feature branch testing, BUT, it is limited support for several SCM providers and even more limited support for self-singed ones make it hard to use in disconnected/self-hosted installation.
So then I went on a journey to create my own pull request Generator using Tekton as my PullRequest listener and 2 new tasks one to create a new branch file in my plugin application and the other to delete those files once the PullRequest is closed.
Let's look at my setup:
Okay, So let's go over that mess of Topology, From left to right:
- On top, we have my local running git server (Gitea, running inside my Openshift instance)
- Under it, we have the Plugin repository that is stored in GitHub.
- In the Openshift Instance, I have created a namespace named “pullrequest” In it I have created the following:
a. a Tekton event-listener
b. a Tekton Template trigger
c. a Tekton Trigger Binding
d. a Tekton Pipeline
e. a Tekton Task — create branch file
f. a Tekton Task — Delete Branch file
g. a Tekton Task — git cli push changes to the repository
h. a Secret with my “.gitconfig” and “.git-credentials” files, that will be mounted to the “git-cli” task. - For the testing, I have set the ArgoCD ApplicationSet that deploys my PR application to this namespace.
- In the Openshift Instance, under the Openshift-GitOps namespace(the default instance), the ApplicationSet generates applications from a GitOps Repository that is hosted in GitHub, with an ArgoCD Application that deploys:
a. the Plug-in Generator application.
b. the tekton YAMLs.
c. the ApplicationSet.
For this Demo, I have 2 application Git Repository that is in the Gitea Git Server, Batman, and Joker.
I have added a webhook to each of those repositories i.e.:
In the Config, I have set the “Pull Request” check box, that sends an event when a pull request is, opened, reopened, or closed, also I checked the Pull request synchronized check box, so the pipeline will be triggered for every change in the commit head in the branch.
Note: I have another pipeline that has an event listener that is webhook to “Gitea” repository and builds a new image for each new commit, so any changes to the git repo will create a new image in the image repository
Ready for some YAMLs?
Let's go over some of the YAML files that are part of this process.
this the the TekTon Pipeline manifest:
apiVersion: tekton.dev/v1beta1
kind: Pipeline
metadata:
name: pull-request
spec:
params:
- default: 'https://github.com/tal-hason/argocd-plugin-generator.git'
name: PLUGIN_REPO
type: string
- default: pull-request
name: PLUGIN_BRANCH
type: string
- default: gh-token
name: TOKEN
type: string
- name: FROM_BRANCH
type: string
- name: NEW_TAG
type: string
- default: open
name: PR_STATE
type: string
- name: PROJECT
type: string
- name: REPO_NAME
type: string
tasks:
- name: get-plugin-repo
params:
- name: url
value: $(params.PLUGIN_REPO)
- name: revision
value: $(params.PLUGIN_BRANCH)
- name: refspec
value: ''
- name: submodules
value: 'true'
- name: depth
value: '1'
- name: sslVerify
value: 'true'
- name: crtFileName
value: ca-bundle.crt
- name: subdirectory
value: ''
- name: sparseCheckoutDirectories
value: ''
- name: deleteExisting
value: 'true'
- name: httpProxy
value: ''
- name: httpsProxy
value: ''
- name: noProxy
value: ''
- name: verbose
value: 'true'
- name: gitInitImage
value: >-
registry.redhat.io/openshift-pipelines/pipelines-git-init-rhel8@sha256:9b14f52b21d29d8d83ea4c0e78623debc954f1a732d2be6d1a7269fbba23b1a4
- name: userHome
value: /home/git
taskRef:
kind: ClusterTask
name: git-clone
workspaces:
- name: output
workspace: workspace
- name: create-app-from-branch
params:
- name: service-name
value: '1234'
- name: git-secret-name
value: $(params.TOKEN)
- name: branch_name
value: $(params.FROM_BRANCH)
- name: tag
value: $(params.NEW_TAG)
- name: repository_name
value: $(params.REPO_NAME)
- name: project_name
value: $(params.PROJECT)
- name: gitops-branch
value: $(params.PLUGIN_BRANCH)
- name: pr-state
value: $(params.PR_STATE)
runAfter:
- get-plugin-repo
taskRef:
kind: Task
name: create-app-from-branch
when:
- input: $(params.PR_STATE)
operator: in
values:
- opened
- reopened
- synchronized
workspaces:
- name: output
workspace: workspace
- name: delete-app-from-branch
params:
- name: service-name
value: '1234'
- name: git-secret-name
value: $(params.TOKEN)
- name: branch_name
value: $(params.FROM_BRANCH)
- name: tag
value: $(params.NEW_TAG)
- name: repository_name
value: $(params.REPO_NAME)
- name: project_name
value: $(params.PROJECT)
- name: gitops-branch
value: $(params.PLUGIN_BRANCH)
- name: pr-state
value: $(params.PR_STATE)
runAfter:
- get-plugin-repo
taskRef:
kind: Task
name: delete-app-from-branch
when:
- input: $(params.PR_STATE)
operator: in
values:
- closed
workspaces:
- name: output
workspace: workspace
- name: git-cli
params:
- name: BASE_IMAGE
value: >-
cgr.dev/chainguard/git:root-2.39@sha256:7759f87050dd8bacabe61354d75ccd7f864d6b6f8ec42697db7159eccd491139
- name: GIT_USER_NAME
value: tal-hason
- name: GIT_USER_EMAIL
value: thason@redhat.com
- name: GIT_SCRIPT
value: |
git config --global --add safe.directory $(workspaces.source.path)
git add -A
git commit -m "Updated changes to Generator"
git push
- name: USER_HOME
value: /root
- name: VERBOSE
value: 'true'
runAfter:
- delete-app-from-branch
- create-app-from-branch
taskRef:
kind: ClusterTask
name: git-cli
workspaces:
- name: source
workspace: workspace
- name: basic-auth
workspace: git-creds
workspaces:
- name: workspace
- name: git-credsthe event-listener configuration:
here there is a trim to 7 characters for the commit sha.
apiVersion: triggers.tekton.dev/v1alpha1
kind: EventListener
metadata:
name: gitea-eventlistener
spec:
namespaceSelector: {}
resources: {}
serviceAccountName: pipeline
triggers:
- bindings:
- kind: TriggerBinding
ref: pull-reqeust-binding
interceptors:
- params:
- name: overlays
value:
- expression: body.pull_request.head.sha.truncate(7)
key: truncated_sha
ref:
kind: ClusterInterceptor
name: cel
name: pull-request
template:
ref: pull-request-templateThe trigger template:
apiVersion: triggers.tekton.dev/v1alpha1
kind: TriggerTemplate
metadata:
name: pull-request-template
spec:
params:
- description: Pull Request state
name: pr_state
- description: The Url of the source repo
name: repo_url
- description: the branch name source from the PR
name: from_branch
- description: the sha of the head of the branch
name: commit_sha
- description: The name of the repository
name: repo_name
- description: The full name of the repository
name: repo_full_name
- description: The name of the orgenization
name: repo_org_name
- description: the 7 first charecters of the commit head
name: truncated_sha
resourcetemplates:
- apiVersion: tekton.dev/v1beta1
kind: PipelineRun
metadata:
generateName: pull-reqeust-from-$(tt.params.from_branch)-
spec:
params:
- name: PLUGIN_REPO
value: 'https://github.com/tal-hason/argocd-plugin-generator.git'
- name: PLUGIN_BRANCH
value: pull-request
- name: FROM_BRANCH
value: $(tt.params.from_branch)
- name: NEW_TAG
value: $(tt.params.truncated_sha)
- name: PR_STATE
value: $(tt.params.pr_state)
- name: PROJECT
value: $(tt.params.repo_org_name)
- name: REPO_NAME
value: $(tt.params.repo_name)
- name: TOKEN
value: gh-token
pipelineRef:
name: pull-request
serviceAccountName: pipeline
timeouts:
pipeline: 1h0m0s
workspaces:
- name: workspace
volumeClaimTemplate:
metadata:
creationTimestamp: null
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 150Mi
storageClassName: crc-csi-hostpath-provisioner
volumeMode: Filesystem
status: {}
- name: git-creds
secret:
secretName: gh-credsand the 2 tasks that I have created for this process:
- the create new branch task:
apiVersion: tekton.dev/v1beta1
kind: Task
metadata:
name: create-app-from-branch
spec:
params:
- default: 'xxx'
name: branch_name
type: string
- default: 'xxx'
name: tag
type: string
- default: 'xxx'
name: repository_name
type: string
- default: 'xxx'
name: project_name
type: string
- default: 'xxx'
name: gitops-branch
type: string
steps:
- args:
- '-c'
- >
set -ex
git config --global --add safe.directory $(workspaces.output.path)
git switch $(params.gitops-branch)
git pull
# Run commands for opened pull request
echo "PR is open. Running script for open PR."
export BRANCH=$(params.branch_name)
export BRANCH=$(echo ${BRANCH} | awk '{print tolower($0)}')
export REPO=$(params.repository_name)
export REPO=$(echo ${REPO} | awk '{print tolower($0)}')
export PROJECT=$(params.project_name)
export PROJECT=$(echo ${PROJECT} | awk '{print tolower($0)}')
export DATETIME=$(date +"%Y-%m-%d_%H:%M:%S")
pwd
cd $(workspaces.output.path)/GitOps/Argo-Plugin/ApplicationFiles
echo "GenerateApplication:" > $REPO-$BRANCH.yaml
echo " branch: ${BRANCH}" >> $REPO-$BRANCH.yaml
echo " tag: $(params.tag)" >> $REPO-$BRANCH.yaml
echo " repoName: ${REPO}" >> $REPO-$BRANCH.yaml
echo " projectName: ${PROJECT}" >> $REPO-$BRANCH.yaml
echo " dateTime: ${DATETIME}" >> $REPO-$BRANCH.yaml
cat $REPO-$BRANCH.yaml
echo "Updating the Plugin Repo"
command:
- /bin/bash
image: 'quay.io/argocicd/update-deploy:pipeline'
name: update-chart-app-ver
workingDir: /workspace/output/
resources: {}
workspaces:
- description: The git repo will be cloned onto the volume backing this Workspace.
name: output2. the delete branch file:
apiVersion: tekton.dev/v1beta1
kind: Task
metadata:
name: delete-app-from-branch
spec:
params:
- default: 'xxx'
name: branch_name
type: string
- default: 'xxx'
name: repository_name
type: string
- default: 'xxx'
name: project_name
type: string
- default: 'xxx'
name: gitops-branch
type: string
steps:
- args:
- '-c'
- >
set -ex
git config --global --add safe.directory $(workspaces.output.path)
git switch $(params.gitops-branch)
git pull
# Run commands for closed pull request
echo "PR is closed. Running script for closed PR."
export BRANCH=$(params.branch_name)
export BRANCH=$(echo ${BRANCH} | awk '{print tolower($0)}')
export REPO=$(params.repository_name)
export REPO=$(echo ${REPO} | awk '{print tolower($0)}')
export PROJECT=$(params.project_name)
export PROJECT=$(echo ${PROJECT} | awk '{print tolower($0)}')
pwd
cd $(workspaces.output.path)/GitOps/Argo-Plugin/ApplicationFiles
ls -l
rm $REPO-$BRANCH.yaml
echo "Delete PR Branch from the Plugin Repo"
command:
- /bin/bash
image: 'quay.io/argocicd/update-deploy:pipeline'
name: update-chart-app-ver
workingDir: /workspace/output/
resources: {}
workspaces:
- description: The git repo will be cloned onto the volume backing this Workspace.
name: outputto access all the files you can go to my repo Plugin-Generator Branch: pull-request.
So Demo time 🤓:
you can add a webhook to ArgoCD so that after each push it will sync the relevant applications and applicationSet from the repositories, in the demo i will refresh the applications manually.
Feel Free to Clone or fork the repo and try and use it, open issues, and contact me if you have any questions or need any assistance.
Links:
- the Git Repository — Here, Please note that you are in the pull-request branch.
- My previous article about the Plugin Generator — Here
- ArgoCD Plugin Generator Manual — Here
- ArgoCD Webhooks, For the Applications — Here For ApplicationSet — Here
Have Fun 🥳
