Parallel Build with Jenkins Pipeline!

In this post, I am focusing on the parallel build in Jenkins declarative pipeline. Hence you trigger your build system by Jenkins there are might be some steps that could run at the same time because they have no dependencies. In this way, you will speed up your build process and save time for other sequential steps.

In the picture above you see there are two main stages QA (Quality Assurance) and DEPLOYMENT. The QA stage runs three jobs at the same time and then starts the Deployment stage with two parallel jobs. I am using Jenkins plugin Blue ocean to get a better-visualized environment.

Tools

* Jenkins Blue ocean plugin
* Jenkins
* docker
Project specific Tools
-> cmake 3.17
-> Clang 10
-> NinjaBuild
-> Conan Artifactory

My project is a simple c++ project and I am using docker as the development environment. Jenkins will start the docker container, clone resources from GitHub, and perform the stages in Jenkinsfile.

The project structure is as below:

├── CMakeLists.txt
├── Jenkinsfile
├── LICENSE
├── README.md
├── src
│ ├── CMakeLists.txt
│ └── main.cpp
└── test
 ├── CMakeLists.txt
 └── test.cpp

The essential part is the Jenkinsfile where we define the declarative pipeline. Jenkins allows us to define sets of steps and stages.

pipeline {
  environment {
   CONAN_USER_HOME = "${env.WORKSPACE}/"
   CONAN_NON_INTERACTIVE = 1
   CONAN_USER_NAME="test"
   CONAN_PASSWORD="12345678" //Dummy password
     }
// here will declera prebuild docker image with volume
   agent { 
      docker {
      image 'runtime-tooling'
      args '-v ${PWD}:/app -w :/app:ro' //read-only
      reuseNode true
      }
    }
//The first stage QA
  stages {
    stage('QA') {
         parallel {
// The substages
           stage('Release') {
            steps {
// We can give command as work with terminal on linux
             sh "mkdir -p /tmp/build-release \
             && cd /tmp/build-release \
             && cmake -DCMAKE_BUILD_TYPE=Release  /var/lib/jenkins/workspace/${env.JOB_NAME} && cmake --build .  \ 
          }
        }
  
  stage('Debug') {
     steps {
       sh "mkdir -p /tmp/build-debug \
       && cd /tmp/build-debug \
       && cmake  -DCMAKE_BUILD_TYPE=Debug /var/lib/jenkins/workspace/${env.JOB_NAME} && cmake --build .  \
      }
    }
stage('Tests') {
   steps {
      sh "mkdir -p /tmp/build-test \
      && cd /tmp/build-test \
      && cmake /var/lib/jenkins/workspace/${env.JOB_NAME} &&  cmake --build .\
      && cd /tmp/build-test/bin/ \
      && ./projectname-tests "
      }
    }
   }
    post {
        failure {
           echo 'This build has failed. See logs for details.'
         }
      }
   }
   stage('DEPLOYMENT'){
      parallel {
        stage('Result') {
          steps {
sh 'echo Reporting...'
             }
          }
  stage('Deploy Conan Artifacts') {
      steps {
         sh "conan remote add project-local "
         sh 'conan upload "" -r=project-local -c'
        }
      }
     }
    }
  }
}

If one of the steps fails then Jenkins will break the rest of the jobs and the build process will breakdown. Now the Jenkinsfile is ready let’s create a Jenkins job via Jenkins UI.

Go to Jenkins UI and

New Item -> enter project name -> select pipeline -> OK

On the next screen simply scroll down to the Pipeline section.

Here either paste your declarative pipeline here or in your Git repository create in Jenkinsfile. I have a repo where all my project sources and the Jenkinsfile located. My step will be as follow:

1. From Definition -> scroll down --> Pipeline script from SCM
2. SCM -> Git
3. Paste your repository URL
4. Apply & Save

Then you will be back to the project dashboard. Hit the “Build Now” and then you can see nice UI via the “Open Blue Ocean” plugin to see visualized steps.

Dashboard

Jenkins will clone the repository from GitHub and check explicitly for Jenkinsfile and then perform steps. If your pipeline fails then check Jenkins logs. The logs are very handy tools to find out where the problem occurs.

Blue Ocean plugin

Or the Jenkins dashboard looks:

Pipeline dashboard

NOTES:
* You can build your docker image or use existing one or get from dockerhub
* You can run jenkins on localhost or in docker container
* ENVIRONMENT VARIABLE could be also stage based. ENV_VAR can be defined for specific stage as well. 

I aimed to give you a simple template for Jenkins's parallel execution using the pipeline. Sure each project is individual which forces you to make your own customizations. I hope this will help you!