GitLab integration with GCP cloud build for CI/CD

In this blog, we will discuss how to integrate Cloud Build with GitLab using webhook triggers, build the Docker image, and deploy it to the Cloud Run.

Problem Statement:

One of our customers wanted to implement CI/CD pipeline using Cloud Build to deploy the Docker Image on Cloud Run. They were having their application code hosted on GitLab. Cloud build provides direct integration with Cloud Source Repository, GitHub, and BitBucket, but there is no straightforward integration available with GitLab.

Integration:

To create a webhook trigger to build on GitLab, we need to create an SSH key that will authenticate your connection to GitLab. You’ll need to retrieve your SSH key in your inline build configuration to access your code on GitLab.

The steps involved in integrating Cloud Build with GitLab are as follow:

1. Create SSH Key.
2. Adding your public SSH access key on GitLab.
3. Storing your private SSH key in Secret Manager.
4. Creating Webhook trigger.
5. Creating a webhook in GitLab.

Implementation:

Step 1: Create SSH Key

Create a new GitLab SSH key, where “gitlab.com” is the URL for your GitLab repository.

ssh-keygen -t rsa -b 4096 -N ‘’ -C gitlab.com -f id_gitlab

This will generate two key files, one is a public file “id_gitlab.pub” and the other is a private key file “id_gitlab”.

Step 2: Adding your public SSH access key on GitLab

Now, we need to add a public SSH key (.pub) which is generated in the first step to GitLab for secure operations. For this follow the below steps.

• Sign in to GitLab.
• Select Preferences, on the top bar, in the top right corner.
• Select SSH Keys, on the left sidebar.
• Paste the contents of your public key in the Key box. If you manually copied the key, make sure you copy the entire key, which starts with ssh-rsa, ssh-dss, ssh-ed25519, ecdsa-sha2-nistp256, ecdsa-sha2-nistp521, sk-ecdsa-sha2-nistp256@openssh.com, or sk-ssh-ed25519@openssh.com, ecdsa-sha2-nistp384 and may end with a comment.
• In the Title box, type a description, like Cloud Build or Cloud Build CI.
• In the Expires date box, select an expiration date (Optional).

Step 3: Storing your private SSH key in Secret Manager

Now we need to store the secret SSH key which is generated in step 1 in the Secret Manager in GCP.

• Go to the Secret Manager page in the Cloud console
• On the Secret Manager page, click Create Secret.
• On the Create secret page, under Name, enter a name for your secret, say gitlab-ssh-secret-key.
• In the Secret value field, paste the contents of your private key.
• Leave the Regions section unchanged.
• Click the Create secret button to create your secret.

Step 4: Creating webhook triggers

Now it's time to create a webhook trigger that invokes builds from GitLab.

• Open the Triggers page.
• Click Create trigger.
• Enter the following trigger settings:

i) Name: A name for your trigger.

ii) Region: Default is set as global. Keep it as is.

iii) Description (Optional): A description of your trigger.

iv) Event: Select the Webhook event to set up your trigger to start builds in response to incoming webhook events.

v) Webhook URL: Use the webhook URL to authenticate incoming webhook events.

• Secret: You will need a secret to authenticate incoming webhook events. You can create a new secret or use an existing secret.

To create a new secret:

a. Select Create New.

b. Click Create Secret.
You will see the Create a webhook secret pop-up box.

c. In the Secret name field, enter a name for your secret, say gitlab-webhook-secret.

d. Click Create secret to save your secret, which will automatically be created and stored for you in Secret Manager

To use an existing secret:

a. Select Use existing.

b. In the Secret field, select the name of the secret you want to use from the drop-down menu or follow the instructions to add a secret by resource ID.

c. In the Secret version field, select your secret version from the drop-down menu.

After you have created or selected your secret, you will see a Webhook URL preview. Your URL will contain an API key generated by Cloud Build and your secret.

• Source (optional): The repository to build when the webhook trigger runs. Leave this field blank.
• Configuration: Create an inline build config in the Google Cloud console.

In the inline YAML file add the three steps as given below, the first two steps in the inline build config first authenticate your connection to GitLab using your SSH key and access your specified repository. The third step checks out the specific commit that invoked this build.

steps:
# first, setup SSH:
# 1- save the SSH key from Secret Manager to a file
# 2- add the host key to the known_hosts file
- name: gcr.io/cloud-builders/git
  args:
  - ‘-c’
  - |
    echo “$$SSHKEY” > /root/.ssh/id_rsa
    chmod 400 /root/.ssh/id_rsa
    ssh-keyscan gitlab.com > /root/.ssh/known_hosts
  entrypoint: bash
  secretEnv:
  - SSHKEY
  volumes:
  - name: ssh
    path: /root/.ssh
# second, clone the repository
- name: gcr.io/cloud-builders/git
  args:
  - clone
  - ‘-n’
  - ‘git@gitlab.com:pranavdhopey/springapp.git’
  - .
  volumes:
  - name: ssh
    path: /root/.ssh
# third, checkout the specific commit that invoked this build
- name: gcr.io/cloud-builders/git
  args:
  - checkout
  - $_TO_SHA
availableSecrets:
  secretManager:
  - versionName: projects/<project_no>/secrets/<secret-name>/versions/latest
  env: SSHKEY

• Substitutions (optional): we can define trigger-specific substitution variables using this field.

In our example, we have created two substitutions variables ‘_BRANCH’ and ‘_TO_SHA’. let’s say you want to watch for a specific branch name associated with a commit ID and then switch to that branch name in the build definition. To obtain this data, you can create substitution variables using payload bindings to save the branch name.

‘_BRANCH’: $(body.ref)
‘_TO_SHA’: $(body.after)

• Filters (optional): You can create a rule within a trigger that determines whether or not your trigger will execute a build based on your substitution variables.

Since we wanted the trigger to execute a build if the branch name matches dev, we have used the “==” operator to check for exact matches.

“_BRANCH == refs/heads/dev”

• Click Create to create your build trigger.

Step 5: Creating a webhook in GitLab

Now in this step, we will be adding the Webhook URL generated in the above step to the GitLab Project.

1. In your project or group, on the left sidebar, select Settings > Webhooks.
2. In the URL, enter the URL of the webhook endpoint. The URL must be percent-encoded if it contains one or more special characters.
3. In the Trigger section, select the events to trigger the webhook.
4. Optional. Clear the Enable SSL verification checkbox to disable SSL verification.
5. Select Add webhook

This is how we can integrate cloud build with GitLab and set up CI/CD pipeline.

Let's understand this with an example. We have a spring-boot application hosted in the GitLab repository. You can copy a sample code from my GitHub repository and host it in your GitLab.

Now follow the above steps to integrate your GitLab repository with Cloud Build and Configure the trigger as specified in the steps.

Workflow:

1. Continuous Integration(CI Pipeline):

• A Developer makes the changes in the code, fixes the bugs, and pushes it to the application repository.
• Cloud build then invokes triggers automatically by the webhook event on the repository.
• Once the trigger gets invoked by any events, cloud build then executes the instructions written in the inline editor such as building the docker image from Dockerfile provided and pushing it to the container registry configured.

2. Continuous Deployment(CD Pipeline)

• Once the Docker image got pushed to the container registry, the cloud build will deploy the image to the cloud run with the given configuration, and finally, the cloud run will host the application on a serverless platform.

Inside the inline editor of the cloud build trigger copy the below code.

In the first step, the inline build config authenticates your connection to GitLab using your SSH key and accesses your specified repository. Following that, in the second and third steps, it clones the repository and checks out the commit that invoked the webhook based on the filter you set on the trigger.

The fourth step of the above inline build config file will build a docker image from the code with the tag as ‘latest’ and with ‘commit id’ extracted from the substitution variable `_TO_SHA` and push the image to the container registry. Now in the last step, the docker image will be deployed to the cloud run with the provided configurations and creates a cloud run service.

Whenever the developer makes the changes, the cloud build trigger gets invoked by the webhook event and executes the steps from the inline build config file which will build the docker image named “springapp” with ‘latest’ and ‘commit id’ as a tag and push it to the GCR.

After the image gets pushed, it will get deployed on a cloud-run service and will get a trigger URL to test it out.

Below is the screenshot of the cloud run trigger URL.

Conclusion:

We have seen the end-to-end deployment of Docker Image on Cloud Run using Cloud Build as a CI/CD tool integrated with the external source code management system GitLab and hosted a sample spring boot application.