How to setup CI/CD pipeline from Jenkins to Gitlab

How Jenkins pipeline works for STORM(internal) project

STORM Jenkins CI/CD Architecture

What is Jenkins?

Jenkins is an open-source automation tool that allows developers all over the world to build, test and deploy their software. It has plenty of plugins in it we will need to download according to our software needs and add the configuration that all are required to run our pipeline.

How we were building docker image in Jenkins

In Jenkins, we were building the docker image with help of Source-to-Image (S2I)(replaced with dockerfile in Gitlab CI/CD pipeline).

The main component that we need to run the pipeline in Jenkins is “Jenkinsfile”.

Jenkinsfile

“Jenkinsfile” is a text file that has all the definition of pipeline and is present in the project's root directory. It can be of the scripted type or declarative type. To know more about “Jenkinsfile” and its type, go to the below link.

Declarative vs. scripted pipelines: What's the difference?

Let’s see how we have written the basic “Jenkinsfile” for our project:

In the above file, we have used the declarative type Jenkinsfile. We have defined three stages:

• Build
• Test
• Deploy

In the build stage, we have given a list of commands to build our application. Here we have shown some sample commands to get the version of java, maven, and Gradle.

In the test stage, we can give commands related to our project to test the application and run the test cases.

And in the deploy stage, we can give the commands that would deploy our application to OpenShift or Kubernetes, whichever you prefer. We were using OpenShift, so gave the series of commands to log in and deploy the application in OpenShift.

We can also set up an optional email notification that would send mail to the mentioned recipient if the pipeline fails.

Sample Jenkins pipeline

Later on, we migrated from Jenkins to Gitlab CI/CD. Gitlab CI/CD syntax is totally different from Jenkins.

Why we have moved our Pipeline to Gitlab CI/CD from Jenkins?

GitLab CI/CD is one of the most loved CI/CD tools used for DevOps testing. GitLab CI/CD has strong documentation, easy control, and a good User Experience on its side. GitLab provides more than what Jenkins is hoping to evolve to, by providing a fully integrated single application for the entire DevOps lifecycle. More than Jenkins’ goals, GitLab also provides planning, SCM, packaging, release, configuration, and monitoring (in addition to the CI/CD that Jenkins is focused on).

With Gitlab, there is no need for plugins and customization. Unlike Jenkins, GitLab is open-source and anyone can contribute changes directly to the codebase, which once merged would be automatically tested and maintained with every change.

Gitlab is a tool for software development that has the following methodology:

• Continuous Integration
• Continuous Delivery
• Continuous Deployment

Continuous Integration

Gitlab CI is a service that is used to build and test the application/code present in the Git repository whenever any developer pushes the code.

Continuous Deployment

Gitlab CD is a software service that pushes the latest changes of the code in the various deployed environment i.e. dev, test, pre-prod, and production depending on one’s need.

How we have set up an automated CI/CD pipeline with Gitlab for BITT transformation project

BITT Gitlab CI/CD architecture

Setup GitLab-runner

To run the Gitlab pipeline firstly we enabled the GitLab-runner our project. To know more about the runners and how to setup runners in your project, you can navigate the below link.

GitLab Runner | GitLab

In the next step, we have created a “.gitlab-ci.yml” file, which defined all the stages that are required for our application.

Gitlab CI YML File

“.gitlab-ci.yml” is a YAML file that is created in our project’s root directory. Whenever a new code is pushed by the developer in the git repository, the “.gitlab-ci.yml” file automatically runs the stages that are defined in it according to various parameters mentioned in the file. You can check below.

image: nexus-test.dt/repository/hal-docker/gradle:4.10.2

variables:
  GRADLE_OPTS: "-Dhttps.protocols="xx" -Dhttp.proxyHost=xxx.xxx.xxx.xx -Dhttp.proxyPort=xxxx -Dhttps.proxyHost=xxx.xxx.xxx.xx -Dhttps.proxyPort=xxxx -Dhttp.nonProxyHosts=*.dt"
  AMWEB_TEST_IMAGE: nexus-test.dt/repository/hal-docker/${CI_PROJECT_NAME}:$CI_COMMIT_SHORT_SHA
  OPENSHIFT_ENV: https://master-test.dt:6443
  OPENSHIFT_PROJECT: test


stages:
  - build
  - publish
  - package
  - deploy

gradle-build:
  stage: build
  tags:
    - docker_runner 
  script:
        - gradle $GRADLE_OPTS clean bootjar -x test
        - gradle task
  artifacts:
    paths:
      - /builds/AMWeb/build/libs/*.jar
    expire_in: 30 min

gradle-publish:
  stage: publish
  tags:
    - docker_runner
  script:
    - gradle publish

build_docker_AMWeb:
  stage: package
  script:
  - docker build -t ${AMWEB_TEST_IMAGE} -f DevOps/Docker/am-web-Dockerfile .
  - docker login -u $NEXUS_USER -p $NEXUS_PASSWORD nexus-test.dt
  - docker push ${AMWEB_TEST_IMAGE}
  tags:
    - docker_runner

deploy_test:
  stage: deploy
  only:
    - tags
  image: appuio/oc:v4.6
  before_script:
    - oc login --server=$OPENSHIFT_ENV --token=$OPENSHIFT_TOKEN --insecure-skip-tls-verify
  script: 
    - oc project $OPENSHIFT_PROJECT
    - echo "Applying Helm "
    - helm upgrade --install am-web DevOps/OpenShift/helm -f DevOps/OpenShift/helm/values-amweb.yaml --set image.tag=latest
  tags:
    - docker_runner

In the above “.gitlab-ci.yml” file, we had defined all the parameters and stages that we need to build, test and deploy our application.

1. At first we have defined the base image that we will be needing to run the application. Since the application is Gradle one so I am using gradle:4.10.2 as a base image to run.

2. Then we are defining the variables in the pipeline that we need to later in the stage and also to keep the pipeline look clean, we have to define the variables in one place.

3. In Gitlab CI/CD you need to define all the stages that you want to run at beginning of the pipeline within the block name “stages”. We need to add stages name in form of a list and the stages will run in the same sequence if no dependency is defined.

4. Then we have given stage definition in all the stages like build, publish, deploy, and script defined what we need to do in a respective stage.

5. Only: tag: By default, the stage runs for every commit but in the case of Only: tag our stage will execute when a new tag has been created on the branch.

6. Tags: — docker_runner: Defined in each stage is the name of the GitLab runner on which pipeline will be running. We need to have GitLab runner enabled for the project to use.

7. To give instruction we use parameters as before_script, script, and after_script. We have used before_script to run the list of commands before the main script. for example any login command etc. In the script, we provide the list of commands that are required for our stages to run, and in after_script is the list of commands that would run after the main script commands have been successfully executed.

8. artifacts: paths We can store the artifacts that are generated after building any application and store them by giving the path and also we can mention the expiration time, till when the artifacts last in storage.

These are the basic steps and parameters we use to set up the pipeline and run and deploy our application.

After creating the GitLab ci file, now whenever any developer pushes changes to the git repository pipeline will trigger, running the respective stages and commands mentioned in them.

Steps to check the status of pipeline and job logs:

1. Go to CI/CD Pipelines. See the below image.

2. Then click on the particular pipeline to check the pipeline status. (like click on #20672).

3. You can also check the console output by clicking on a particular stage, to check the build log click on the “Build” stage.

Check Junit test cases reports by clicking on the “Tests” section.

To add any plugin or integrate anything in GitLab is far easier than Jenkins. To integrate anything in Jenkins we need to add the plugins and then integrate it, but GitLab has provided all just with one click. It has made it very easy and user-friendly.

I have added email integration in the pipeline, that for failure/success I should get an email. We just need to go to Settings  Integration Pipeline emails  Add Recipients. See the below image.

In the same way, we can add more integration and webhooks, that are required in our projects.

This is the basic setup we did to migrate our pipeline from Jenkins to Gitlab.

Hope you enjoyed reading this blog.

🚀 If you found insights into setting up a CI/CD pipeline from Jenkins to GitLab and understanding how the Jenkins pipeline works for the STORM(internal) project valuable, stay tuned for more DevOps and CI/CD insights.

👏 Liked what you read? Give it a round of applause! Your support fuels our commitment to delivering informative content.

🔗 Found this article beneficial? Share it with your network, and let’s empower more developers with efficient CI/CD practices!

🤔 How has your experience been with CI/CD tools like Jenkins and GitLab? Share your thoughts or questions in the comments below! Your engagement sparks valuable conversations.