Writing a Resusable CICD Pipeline using OpenShift Pipelines

Writing and Running a ‘Source to Image to Deploy Pipeline’

6 min readDec 6, 2019

Our goal in this article is to build a cloud native CICD pipeline for a ‘Voting Web App’. The application consists of 2 components a frontend service and a backend service. We will be setting up a CICD pipeline which has 2 stages:

Building container images (using respective Dockerfiles)
Deploying application on OpenShift (Deployments, Services)

Prerequisites

An OpenShift 4.x cluster (CRC, AWS or any OpenShift 4.x Cluster)
Install OpenShift Pipelines (part 1: How to Install OpenShift-Pipelines on OpenShift 4.x)
Install Tetkon CLI ‘tkn’ on your local machine: https://github.com/tektoncd/cli#installing-tkn

Our ‘Voting’ Web-Application

We will setting up a reusable CICD pipeline using OpenShift Pipeline to build and deploy a voting application on OpenShift.

This application has a frontend (Web UI) powered by python and a backend (Datastore) written in Golang. The the source code repositories are:

Frontend WebUI: https://github.com/nikhil-thomas/vote-ui
Backend Datastore: https://github.com/nikhil-thomas/vote-api

Fork these repositories so that you can have full control over the application source code. The forks will enable you to push patches to the source code and rerun the CICD pipeline.

CICD Pipeline Definition

The pipeline we are building is there in the ‘Vote-CICD’ repository:

Vote-CICD: https://github.com/nikhil-thomas/vote-cicd

Fork this repository in github and clone your fork on your local machine.

$ git clone https://github.com/<your-github-username>/vote-cicd.git

Go to your Vote-CICD clone location on your local machine and checkout `part-2-cicd-pipeline` branch

$ cd vote-cicdvote-cicd$ git checkout -b part-2-cicd-pipeline origin/part-2-cicd-pipeline

The directory structure will be:

vote-cicd
├── 02-pipelineresources
│   ├── 01-vote-api-git-resource.yaml
│   ├── 02-vote-api-image-resource.yaml
│   ├── 04-vote-ui-git-resource.yaml
│   └── 04-vote-ui-image-resource.yaml
├── 03-tasks
│   └── 01-oc-apply-deployment-task.yaml
├── 04-pipelines
│   └── 01-build-deploy-pipeline.yaml
├── 05-pipelineruns
│   ├── 01-build-deploy-api-pipelinerun.yaml
│   └── 02-build-deploy-ui-pipelinerun.yaml
├── LICENSE
└── README.md

Create a New Project

vote-cicd$ oc new-project cote-cicdNow using project "vote-cicd" on server ...
...

Creating PipelineResource

We need to create 4 Pipeline Resources:

vote-api source code (PipelineResource type: git)
vote-api container image(PipelineResource type: image)
vote-ui source code (PipelineResource type: git)
vote-ui container image (PipelineResource type: image)

The PipelineResource definitions are in the 02-pipelineresources directory. . If you are using your fork of `vote-api` and `vote-ui ` use your github repository urls instead of nikhil-thomas/vote-api, edit the vote-api-git-resource.yaml and vote-ui-git-resource.yaml.

apiVersion: tekton.dev/v1alpha1
kind: PipelineResource
metadata:
  name: vote-api-repo
spec:
  type: git
  params:
  - name: url
    value: https://github.com/<your github username>/vote-api.git

Now add the PipelineResources to your cluster using the oc command.

vote-cicd$ oc apply -f 02-pipelineresources/pipelineresource.tekton.dev/vote-api-repo created
pipelineresource.tekton.dev/vote-api-image created
pipelineresource.tekton.dev/vote-ui-repo created
pipelineresource.tekton.dev/vote-ui-image created

CICD Pipeline Tasks

Our CICD Pipeline has two stages:

Build Container Image: We will be using the default `buildah-v0–8–0` Task (ClusterTask) shipped with the OpenShift-Pipelines-Operator
Deploy Container Image on OpenShift: We will be writing a custom task to accomplish this.

The Task`oc-apply-deployment` is defined in: 03-tasks . Add it to the cluster.

vote-cicd$ oc apply -f 03-tasks/task.tekton.dev/oc-apply-deployment created

Creating Build-Deploy Pipeline

Add the Pipeline defined in 04-pipelines

vote-cicd$ oc apply -f 04-pipelines/pipeline.tekton.dev/build-and-deploy created

DevConsole and our CICD Pipeline

On the OpenShift 4.x WebConsole switch to Developer Perspective by clicking the drop-down menu in the top left corner. Select project: vote-cicd. Our `build-and-deploy` pipeline will be listed there.

We can see that the pipeline has 2 stages:

build-image: Task-ref: buildah-v0–8–0 (ClusterTask supplied by operator)
create-update-deployment: TaskRef: oc-apply-deployment (Task created by us)

The individual steps within tasks will be shown in a pop up when we hover the mouse pointer on a Task in the Pipeline.

Build and Deploy Voting App

The same build-and-deploy Pipeline can be used to build vote-api and vote-ui. The vote-api and vote-ui are built and deployed by creating PipelineRuns. PipelineRuns bind a Pipeline to a projects PipelineResources (src-repo, image) and necessary input parameters. We do it by creating separate PipelineRuns for vote-api and vote-ui projects as defined in 05-pipelineruns.

Start the Build-and-Deploy PipelineRuns for vote-api and vote-ui

vote-cicd$ oc apply -f 05-pipelineruns/pipelinerun.tekton.dev/build-deploy-api-pipelinerun created
pipelinerun.tekton.dev/build-deploy-ui-pipelinerun created

Using Tekton Command Line Interface tkn we can list Tekton resources. We can fetch well formatted logs from PipelineRuns.

We can look at the PipelineRun logs using tkn pr logs -f <pipelinerun-name> command.

In addition, the PipelineRuns and their logs can be viewed from the DevConsole UI as well in DeveloperPerspective.

Verifying Vote-Application

On successful completion of both PipelineRuns we should see:

Deployments: 1 each for vote-ui and vote-api
Services: 1 each for vote-ui and vote-api
Route: 1 for vote-ui (frontend)

The Vote application will be available on the newly created route.

ReRunning CICD Pipeline

First we have to push updates Vote-UI or Vote-API source code to respective git repositories. The we can just rerun the PipelineRuns. It will fetch the new updates build a new container image and then update the images in the respective (frontend/backend) deployments.

References:

Tekton Pipelines Overview: https://www.youtube.com/watch?v=-ji5Z0qJmJs
Tekton Pipelines Getting Started: https://github.com/tektoncd/pipeline/tree/master/docs#tekton-pipelines
Tekton Pipelines Examples: https://github.com/tektoncd/pipeline/tree/master/examples#examples
TektonCD Task Catalog: https://github.com/tektoncd/catalog
TektonCD Github Org: https://github.com/tektoncd
OpensShift Pipelines Tutorial: https://github.com/openshift/pipelines-tutorial