Tekton Pipeline — Kubernetes-native pipelines

Tekton Pipeline is a new project which allows you to run your CI/CD pipelines in a Kubernetes-native approach. Tekton Pipelines emerged out of the Knative build project. If you like to know more about Knative I highly recommend you to visit their project website which is available here.

Before we start talking about what’s Kubernetes-native means and how Tekton Pipelines are working I would like to first take a step back and clarify shortly why containerized pipelines are so important and helpful:
Some time ago we started putting our workloads into containers. We did so because of advantages like isolation, dependencies, scalability and immutability. Wouldn’t these be also helpful in your CI/CD pipeline?
Think of “build-hosts” which only provide the tools and dependencies needed for one particular pipeline task. An environment which is looking the same every run and does not have any dependencies of other projects which might cause issues. As well as easily scalable pipelines.
That’s why we need and should use containerized pipelines!

Now, where we briefly talked about containerized pipelines let’s talk about where Tekton Pipeline with its Kubernetes-native approach can help:
Tekton Pipeline allows us to run our containerized pipelines in our existing Kubernetes Clusters. This means we do not need additional machines to run our pipelines and therefore can better utilize our existing ones.
This is greats but, to be honest, that alone does not make Tekton Pipeline unique. Tekton Pipeline goes one step further and also stores everything related to our pipeline within Kubernetes — as Kubernetes resources. This allows us to work with our pipelines as we do with any other resource. Think of a Deployment or Service which you can create and manage using kubectl and YAML files.

How to get started

As mentioned above Tekton Pipeline lives within a Kubernetes Cluster. It is based on 5 Custom Resource Definitions (CRDs), Deployments, Configmaps and Services. You can run the following command to get started:

kubectl apply -f https://storage.googleapis.com/tekton-releases/latest/release.yaml

Besides the above-mentioned resources, it will also create a Namespace, a Pod Security Policy, a Service Account and ClusterRoles. Tekton Pipeline is ready as soon as all Pods in the newly created Namespace (the default name is tekton-pipelines) are ready.
You can, of course, review the above YAML and customize it based on your needs.

If you need to share artifacts or other pipelines resources between your tasks you will need to configure a storage option. You can either use PVCs which will be requested every time needed (Dynamic Volume Provisioning is key!) or Blob storage. You will find more details on this task here.

A first pipeline

So, how does Tekton Pipelines work? I will explain the different resources (Custom Resource Definitions) of Tekton Pipeline using the small example below. The pipeline will build a small Go application, build the depending Image and then push it into a Registry. You will find all the related files here.

First of all, we will create two PipelineResouce definitions which we will use to provide the source Git repository and the target Registry. The Pipeline Resouces are optional but are very helpful to reuse the same pipeline with different sources and targets.

apiVersion: tekton.dev/v1alpha1
kind: PipelineResource
metadata:
  name: git-repo
spec:
  type: git
  params:
    - name: revision
      value: master
    - name: url
      value: https://gitlab.com/nmeisenzahl/tekton-demo
---
apiVersion: tekton.dev/v1alpha1
kind: PipelineResource
metadata:
  name: image-registry
spec:
  type: image
  params:
    - name: url
      value: registry.gitlab.com/nmeisenzahl/tekton-demo/demo:latest

Now we need to create a Task resource to define the steps of our pipeline. You can, of course, define multiple tasks if you need them. In our example, we will use Kaniko to build our Image. The Dockerfile, as well as the app resources, are stored in the Git repository.

apiVersion: tekton.dev/v1alpha1
kind: Task
metadata:
  name: build-docker-image
spec:
  inputs:
    resources:
      - name: git-repo
        type: git
    params:
      - name: pathToDockerFile
        description: Path to Dockerfile
        default: /workspace/git-repo/Dockerfile
      - name: pathToContext
        description: The build context used by Kaniko
        default: /workspace/git-repo
  outputs:
    resources:
      - name: image-registry
        type: image
  steps:
    - name: build-and-push
      image: gcr.io/kaniko-project/executor:v0.10.0
      env:
        - name: "DOCKER_CONFIG"
          value: "/builder/home/.docker/"
      command:
        - /kaniko/executor
      args:
        - --dockerfile=${inputs.params.pathToDockerFile}
        - --destination=${outputs.resources.image-registry.url}
        - --context=${inputs.params.pathToContext}

We could now create a TaskRun resource to run an instance of the above task. However, in this example, we use the Pipeline which we can use to combine multiple tasks in a pipeline:

apiVersion: tekton.dev/v1alpha1
kind: Pipeline
metadata:
  name: demo-pipeline
spec:
  resources:
    - name: git-repo
      type: git
    - name: image-registry
      type: image
  tasks:
    - name: build-docker-image
      taskRef:
        name: build-docker-image
      params:
        - name: pathToDockerFile
          value: /workspace/git-repo/Dockerfile
        - name: pathToContext
          value: /workspace/git-repo
      resources:
        inputs:
          - name: git-repo
            resource: git-repo
        outputs:
          - name: image-registry
            resource: image-registry

Because we move the image to a registry, you must ensure that the pipeline can authenticate itself by configuring an ImagePullSecrets for the service account used (in our case, the default service account).

We now have everything in place to finally run the pipeline. For that, we need a final definition. A PipelineRun resource:

apiVersion: tekton.dev/v1alpha1
kind: PipelineRun
metadata:
  name: demo-pipeline-run-1
spec:
  pipelineRef:
    name: demo-pipeline
  resources:
    - name: git-repo
      resourceRef:
        name: git-repo
    - name: image-registry
      resourceRef:
        name: image-registry

To verify the status of your pipeline run you can execute kubectl get pipelineruns -oyaml .

There is more

Beside the Tekton Pipeline project itself, there is also a project for a CLI which makes you work with your pipelines easier. You can also install a web-based Dashboard to view and manage your pipelines in a browser.

Furthermore, the team is working on another project called Tekton Triggers. This project is pretty new (the first commit was 4 weeks ago) and still work in progress. Tekton Triggers will allow calling Tekton pipelines based on “triggers”. Those could be git commits, git issues or any kind of webhooks. More details are available here.