Tekton Pipeline — Kubernetes-native pipelines

Nico Meisenzahl
Jul 28 · 4 min read

Tekton Pipeline is a new project which allows you to run your CI/CD pipelines in a Kubernetes-native approach. Tekton Pipelines emerged out of the Knative build project. If you like to know more about Knative I highly recommend you to visit their project website which is available here.

Before we start talking about what’s Kubernetes-native means and how Tekton Pipelines are working I would like to first take a step back and clarify shortly why containerized pipelines are so important and helpful:
Some time ago we started putting our workloads into containers. We did so because of advantages like isolation, dependencies, scalability and immutability. Wouldn’t these be also helpful in your CI/CD pipeline?
Think of “build-hosts” which only provide the tools and dependencies needed for one particular pipeline task. An environment which is looking the same every run and does not have any dependencies of other projects which might cause issues. As well as easily scalable pipelines.
That’s why we need and should use containerized pipelines!

Now, where we briefly talked about containerized pipelines let’s talk about where Tekton Pipeline with its Kubernetes-native approach can help:
Tekton Pipeline allows us to run our containerized pipelines in our existing Kubernetes Clusters. This means we do not need additional machines to run our pipelines and therefore can better utilize our existing ones.
This is greats but, to be honest, that alone does not make Tekton Pipeline unique. Tekton Pipeline goes one step further and also stores everything related to our pipeline within Kubernetes — as Kubernetes resources. This allows us to work with our pipelines as we do with any other resource. Think of a Deployment or Service which you can create and manage using kubectl and YAML files.

How to get started

As mentioned above Tekton Pipeline lives within a Kubernetes Cluster. It is based on 5 Custom Resource Definitions (CRDs), Deployments, Configmaps and Services. You can run the following command to get started:

kubectl apply -f https://storage.googleapis.com/tekton-releases/latest/release.yaml

Besides the above-mentioned resources, it will also create a Namespace, a Pod Security Policy, a Service Account and ClusterRoles. Tekton Pipeline is ready as soon as all Pods in the newly created Namespace (the default name is tekton-pipelines) are ready.
You can, of course, review the above YAML and customize it based on your needs.

If you need to share artifacts or other pipelines resources between your tasks you will need to configure a storage option. You can either use PVCs which will be requested every time needed (Dynamic Volume Provisioning is key!) or Blob storage. You will find more details on this task here.

A first pipeline

So, how does Tekton Pipelines work? I will explain the different resources (Custom Resource Definitions) of Tekton Pipeline using the small example below. The pipeline will build a small Go application, build the depending Image and then push it into a Registry. You will find all the related files here.

First of all, we will create two PipelineResouce definitions which we will use to provide the source Git repository and the target Registry. The Pipeline Resouces are optional but are very helpful to reuse the same pipeline with different sources and targets.

apiVersion: tekton.dev/v1alpha1
kind: PipelineResource
metadata:
name: git-repo
spec:
type: git
params:
- name: revision
value: master
- name: url
value: https://gitlab.com/nmeisenzahl/tekton-demo
---
apiVersion: tekton.dev/v1alpha1
kind: PipelineResource
metadata:
name: image-registry
spec:
type: image
params:
- name: url
value: registry.gitlab.com/nmeisenzahl/tekton-demo/demo:latest

Now we need to create a Task resource to define the steps of our pipeline. You can, of course, define multiple tasks if you need them. In our example, we will use Kaniko to build our Image. The Dockerfile, as well as the app resources, are stored in the Git repository.

apiVersion: tekton.dev/v1alpha1
kind: Task
metadata:
name: build-docker-image
spec:
inputs:
resources:
- name: git-repo
type: git
params:
- name: pathToDockerFile
description: Path to Dockerfile
default: /workspace/git-repo/Dockerfile
- name: pathToContext
description: The build context used by Kaniko
default: /workspace/git-repo
outputs:
resources:
- name: image-registry
type: image
steps:
- name: build-and-push
image: gcr.io/kaniko-project/executor:v0.10.0
env:
- name: "DOCKER_CONFIG"
value: "/builder/home/.docker/"
command:
- /kaniko/executor
args:
- --dockerfile=${inputs.params.pathToDockerFile}
- --destination=${outputs.resources.image-registry.url}
- --context=${inputs.params.pathToContext}

We could now create a TaskRun resource to run an instance of the above task. However, in this example, we use the Pipeline which we can use to combine multiple tasks in a pipeline:

apiVersion: tekton.dev/v1alpha1
kind: Pipeline
metadata:
name: demo-pipeline
spec:
resources:
- name: git-repo
type: git
- name: image-registry
type: image
tasks:
- name: build-docker-image
taskRef:
name: build-docker-image
params:
- name: pathToDockerFile
value: /workspace/git-repo/Dockerfile
- name: pathToContext
value: /workspace/git-repo
resources:
inputs:
- name: git-repo
resource: git-repo
outputs:
- name: image-registry
resource: image-registry

Because we move the image to a registry, you must ensure that the pipeline can authenticate itself by configuring an ImagePullSecrets for the service account used (in our case, the default service account).

We now have everything in place to finally run the pipeline. For that, we need a final definition. A PipelineRun resource:

apiVersion: tekton.dev/v1alpha1
kind: PipelineRun
metadata:
name: demo-pipeline-run-1
spec:
pipelineRef:
name: demo-pipeline
resources:
- name: git-repo
resourceRef:
name: git-repo
- name: image-registry
resourceRef:
name: image-registry

To verify the status of your pipeline run you can execute kubectl get pipelineruns -oyaml .

There is more

Beside the Tekton Pipeline project itself, there is also a project for a CLI which makes you work with your pipelines easier. You can also install a web-based Dashboard to view and manage your pipelines in a browser.

Furthermore, the team is working on another project called Tekton Triggers. This project is pretty new (the first commit was 4 weeks ago) and still work in progress. Tekton Triggers will allow calling Tekton pipelines based on “triggers”. Those could be git commits, git issues or any kind of webhooks. More details are available here.

01001101

Stories related to DevOps topics by Nico Meisenzahl. 01001101? First char of my surname.

Nico Meisenzahl

Written by

Senior Cloud & DevOps Consultant at white duck. Docker Community Leader, GitLab Hero, blogger & speaker. 👨‍💻🙋‍♂️ Loves Kubernetes, DevOps & Cloud.

01001101

01001101

Stories related to DevOps topics by Nico Meisenzahl. 01001101? First char of my surname.

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade