Tekton — multiple GitHub repositories triggering one EventListener

While working on one project, as a DevOps engineer, I was given an interesting task, connect multiple GitHub repositories to one EventListener in Tekton, to serve the CI/CD needs of our project team, composed of backend, frontend, and infrastructure engineers. The project was allocated a Tekton space with a limitation to 2 EventListeners, and other teams were supposed to join the project soon.

As a general practice, and to avoid complicated logic, a Tekton EventListener should listen to one repository and should operate as a pair, allowing the Interceptors & Triggers to respond to specific events from that repository. However, this was not the case for my task.

So let’s see what one way of resolving this task might look like in this article.

Now, I did not set up the environment, I just operate the provided Tekton space, so I don’t have any input on the installation of Tekton or the provisioning of the GitHub secret used in conjunction with the webhook, but I want to point out briefly the components of this ecosystem, using the below diagram as visual:

The high-level view of the general Tekton ecosystem.

From left to right, we have:

• GitHub webhooks — these are pre-configured payload messages, usually in a JSON format, that GitHub can send to other applications based on different events, like a pull_request or a push . For webhooks to be able to send these payloads to your system, you need to provide the URL of a service that can receive and parse the payload, together with a valid Token — if you use Tekton, that service takes the form of a Routeresource;
• Routes act like an ingress, exposing a URL outside of the Openshift or Kubernetes cluster to which webhooks connect and send payloads. Just like an ingress resource, Routes then map to a service inside the cluster that sends the payloads to the EventListenerPod.
• The EventListener defines Triggers and Interceptors that meet GitHub payloads and then, based on filters, the payloads can be scanned, transformed, and forwarded to TriggerBindings.
• TriggerBindings bridge the EventListener to TriggerTemplates and also expose Parameters from the webhook body message to be used throughout the Pipeline execution lifecycle.
• TriggerTemplates define how PipelineRuns will be created to kick off eventual Pipelines automatically, assigning Volumes and populating Parameters with payload information to be used later on by Tasks.
• Tasks are a list of actions within the Pipeline.
• Steps are the atomic component of Tekton and one Task can have one or multiple Steps defined, each Step creating its own Pod, loading the defined Container.

Of course, the discussion is broader, but this should give you an idea of how Tekton operates. We will mostly focus on the first 3 bullet points in this article.

GitHub Webhooks

In GitHub, you need to make sure you are an Admin of the repository you want to link to Tekton. You should have access to the Settings > Hooks section, like so:

I have configured an HTTPS Route to be linked to this webhook and which will be informed based on pull_request and push events, but you can select from a multitude of different event types. A green mark in front of the URL shows that everything is working correctly (the URL is reachable and can be connected via the supplied Tekton secret token).

Note: Sometimes, when adding the URL and the Token, you may see a red X instead of the green checkmark in front of the URL, even after a few minutes, appearing like the route is not working. Try doing something on a branch, like pushing a commit or a similar git action, so that an actual payload will be sent out to your Route. The mark will change from Red to Green (fingers crossed :D).

In terms of configuring the webhook, this is how I set it up, and I’ll go over each point:

Webhook configuration

1. Payload URL — this is where you paste in your Route’s exposed URL (you have a choice of creating an HTTP or HTTPS Route via Tekton);
2. Since we are using Tekton, we’re expecting to receive content of the type application/json so that’s what I’ve selected;
3. In the Secret section, I’ve pasted in my Tekton secret configured for this specific GitHub webhooks integration;
4. Since I've set up an HTTPS-type Route, I leave SSL verification Enabled;
5. Since I only want to handle pull_request and push event types, I select them individually from the list of events.

The Route

The rest of the resources that we need to create will be created on the Tekton side. Since Tekton is basically an Operator in Kubernetes that leverages CRDs, so will our Resources be defined as yaml Kubernetes resource files, and the Route is no exception.

However, although I am using yaml files, you can configure everything from the Tekton Dashboard GUI interface.

Now, in order to expose an HTTPS route, you will need to configure a Domain and have the route created based on your private domain, if you are targeting a Production grade deployment of the solution, and I already have something set up by my admins, so I’ll only showcase how the Route resource looks like.

Note: a Route is not actually a Tekton resource, but is the Openshift ingress resource available out of the box. Clear enough if we look at the apiVersion field.

apiVersion: route.openshift.io/v1
kind: Route
metadata:
  name: my-webhook
  namespace: my-namespace
  labels:
    app: my-cicd-pipeline
    app.kubernetes.io/managed-by: EventListener
    app.kubernetes.io/part-of: Triggers
    eventlistener: my-cicd-eventlistener
  annotations:
    openshift.io/host.generated: 'true'
spec:

  # This will be your exposed webhook URL, prefixed by https://
  host: my-webhook.example.com

  # This service will be created automatically and will link the Route to the EventListener Pod
  to:
    kind: Service
    name: my-webhook-service
    weight: 100

  port:
    targetPort: http-listener

  wildcardPolicy: None

  # This part relates to enabling HTTPS on your Route
  tls:
    termination: edge
    insecureEdgeTerminationPolicy: Redirect

The EventListener

For context, let’s imagine we have 2 teams in the project, each with an individual repository that we will set up a Trigger for:

TEAM      RESOURCE    GITHUB_ORG/GITHUB_REPO  
Team 1    teamone     example-corp/team-one-repo
Team 2    teamtwo     example-corp/team-two-repo

We’ll take a look at the EventListener‘s yaml file next:

apiVersion: triggers.tekton.dev/v1beta1
kind: EventListener
metadata:
  name: my-cicd-eventlistener
  namespace: my-namespace
  labels:
    app: my-cicd-pipeline
spec:
  serviceAccountName: pipeline
  namespaceSelector: {}
  resources:
    kubernetesResource:
      spec:
        template:
          metadata:
            creationTimestamp: null
          spec:
            containers:
              - resources:
                  limits:
                    cpu: 25m
                    ephemeral-storage: 150Mi
                    memory: 96Mi
                  requests:
                    cpu: 5m
                    ephemeral-storage: 75Mi
                    memory: 48Mi
            nodeSelector:
              node-role.kubernetes.io/el: ""
            serviceAccountName: pipeline
            tolerations:
              - effect: NoSchedule
                key: node-role.kubernetes.io/el
                operator: Exists

  # Each trigger suits one team and consequently one team's repository.
  triggers:

    #########################
    #### Team 1 triggers ####
    #########################

    # The pull-request trigger gets fired on opening or updating a pull-request
    - name: teamone-pull-request

      # Bindings reference the TriggerBinding resource the trigger will call on when it is fired
      bindings:
        - kind: TriggerBinding

          # This is another resource I create as a yaml file
          ref: teamone-github-pullrequest-trigger-binding

      # And this references the TriggerTemplate
      template:

        # Which is another resource I create as a yaml file
        ref: teamone-github-pullrequest-trigger-template

      # And then we have the interceptors which do the heavy lifting
      interceptors:

        # It always helps to give them a name
        - name: "Defines the event type"
          ref:
            kind: ClusterInterceptor
            name: github
          params:

            # We pass on the same Token as the one used by GitHub's webhook
            - name: secretRef
              value:
                secretKey: secretToken
                secretName: github-webhooks-secret
    
            # I'm only interested in pull_request events from this team
            - name: eventTypes
              value:
                - pull_request

        - name: "Checks the type of incoming event and fires only on defined types"
          ref:

            # You should reallly read about the `cel` interceptor
            kind: ClusterInterceptor
            name: cel
          params:
            
            # Filters are the way we triage requests coming from different repositories
            #   body.repository.full_name and body.action are parts of the GitHub payload json
            #   .matches('repo_name') is a function to search for a string in the body of the
            #     payload field "full_name" and we map it to the actual repository name
            #   in ['string1','string2'] is how we filter out the GitHub event action type
            - name: filter
              value: >-
                body.repository.full_name.matches('example-corp/team-one-repo') &&
                body.action in ['opened', 'reopened', 'edited', 'synchronize']

            # Overlays are CEL expressions added by the Trigger to the event payload in the
            #  top-level extensions field. Overlays are accessible from TriggerBindings.
            # Here I'm exposing the label key with the value of a transformed repository name,
            #   to use later in the Pipeline.
            - name: overlays
              value:
                - expression: body.repository.full_name.lowerAscii().replace('/','-').replace('.', '-').replace('_', '-')
                  key: label

    

    # And similar for Team 2:

    #########################
    #### Team 2 triggers ####
    #########################

    # The pull-request trigger gets fired on opening or updating a pull-request
    - name: teamtwo-pull-request
      bindings:
        - kind: TriggerBinding
          ref: teamtwo-github-pullrequest-trigger-binding
      template:
        ref: teamtwo-github-pullrequest-trigger-template
      interceptors:

        - name: "Defines the event type"
          ref:
            kind: ClusterInterceptor
            name: github
          params:
            - name: secretRef
              value:
                secretKey: secretToken
                secretName: github-webhooks-secret
            - name: eventTypes
              value:
                - pull_request

        - name: "Checks the type of incoming event and fires only on defined types"
          ref:
            kind: ClusterInterceptor
            name: cel
          params:
            - name: filter
              value: >-
                body.repository.full_name.matches('example-corp/team-two-repo') &&
                body.action in ['opened', 'reopened', 'edited', 'synchronize']
            - name: overlays
              value:
                - expression: body.repository.full_name.lowerAscii().replace('/','-').replace('.', '-').replace('_', '-')
                  key: label

The main thing to note is I use body.repository.full_name.matches(‘org/repo_name’) to run a filter per each repository and link it to other conditions using && or || — Tekton interpreters — and I then provide the logic on how Triggers will fire requests.

The drawback is I have to do this for each filter in particular, duplicating code lines and there are some limitations to these filters, however, for my purpose, this does the job.

If you need to do any troubleshooting on the filters, remember that the EventListener runs under a Pod in Kubernetes, so you can view the log of that Pod and check which Triggers misfire.

Let me know if you have tackled this challenge in a different way, really curious about how I can improve this task further.

Thank you for your read! :)