Configure CI for a Node/Java project in Cloud Foundry using Gitlab Runner

The Problem statement

A node/java project on Gitlab needs CI configured using Gitlab Runner on Pivotal Cloud Foundry.

Gitlab is an efficient repository manager that provides CI/CD pipeline features, hence I choose to push my source code to Gitlab and deploy the Runner on PCF.

Gitlab Runner is a tool on which we run our pipelines.

Architecture : Node/java project + Gitlab Runner + PCF ( for CI ).

Pre-requisite : 
1. Git and Docker installed on your local system
2. Gitlab account
3.Docker Hub account
4.PCF account

Solution :

We need to Create a Project and push to Gitlab

A basic node/Java or any other project. 
eg : https://gitlab.com/debanjana.maitra/pipeline-test


Where do we want our CI to run ?
- Pivotal Cloud Foundry.

Create a PCF organisation and space


We would need to containerise the Runner.
To create a container , we would need a Dockerfile and Entrypoint script.

Create a Dockerfile.

Create an entrypoint script


We would need to Build an Image from the Dockerfile created above.

docker build -t IMAGE-NAME --file PATH_TO_Dockerfile

eg :

docker build -t gitlab-runner-pcf --file Dockerfile .

We should Push the built Image to Docker Hub , so that it is available for deployment on PCF in the next steps.

docker login --username=usernameOfDockerHub
docker tag IMAGE_ID DOCKERHUB_USERNAME/RUNNER-NAME:latest
docker push DOCKERHUB_USERNAME/RUNNER-NAME

Example as shown below:

docker login --username=debanjanamaitra
docker tag cf-gitlab-runner debanjanamaitra/cf-gitlab-runner:latest
docker push debanjanamaitra/cf-gitlab-runner

Finally we need to Deploy the Image On PCF and Register the Runner .

Deploy the image on PCF , but do not start it

Deploying the image on PCF creates an APP in your defined Organisation and Space in PCF.

cf push PCF-APP-NAME --no-start --docker-image DOCKERHUB_USERNAME/RUNNER-NAME:latest --health-check-type process

Example as shown below:

cf push gitlab-runner-pcf --no-start --docker-image debanjanamaitra/cf-gitlab-runner:latest --health-check-type process

Set environment variables

Setting the environment variable , registers the Runner 
REGISTRATION_TOKEN can be obtained from Your project> Settings >CI/CD > Runners settings
API_PRIVATE_TOKEN can be obtained from User settings > Access Token > Generate a new personal access token

cf set-env gitlab-runner-pcf REGISTRATION_TOKEN "REPLACE_WITH_YOUR_PROJECT_RUNNER_REG_TOKEN"
cf set-env gitlab-runner-pcf API_PRIVATE_TOKEN "REPLACE_WITH_GITLAB_SETTINGS_ACCESS_TOKEN"
cf set-env gitlab-runner-pcf API_RUNNER_URL "https://gitlab.com/api/v4/runners"
cf set-env gitlab-runner-pcf CI_SERVER_URL "https://gitlab.com/"
cf set-env
gitlab-runner-pcf RUNNER_NAME "CUSTOM-RUNNER-NAME"
cf set-env
gitlab-runner-pcf REGISTER_NON_INTERACTIVE true
cf set-env
gitlab-runner-pcf REGISTER_RUN_UNTAGGED true
cf set-env
gitlab-runner-pcf RUNNER_EXECUTOR "shell"

Start and restage

cf start gitlab-runner-pcf
cf restage
gitlab-runner-pcf

Let’s Check if the Runner has been registered.

Navigate to Settings > CI/CD > Runners . Check if the runner with the name set ( in environment variable) has been registered.

Make sure to cross check the Runner name in PCF logs


Once the Runner has been registered , what do we run on the Runner?

A Pipeline.

What do we need to define a pipeline?
- A pipeline descriptor script.

Add a pipeline script (.gitlab-ci.yml) that defines your stages and jobs in each stage.
Here I have a very simple script that has one stage called test and in that I run an npm production build for my project.


Run pipeline

Note:

If your project has large number of dependencies , the runner might need more than default space/memory (which in my case for 1 GB). If there is limited memory , the pipeline might fail. Check the error in logs and if required , increase the memory in PCF > Application > Settings> Scaling.