Creating a DevSecOps pipeline with Jenkins — Part 2

Hello everyone. I’m Ata, a computer science graduate and currently interested in cybersecurity. In the fast-paced world of cybersecurity, staying ahead of potential threats is crucial. As a junior cybersecurity engineer with a specific interest in DevSecOps, I am excited to share my journey in creating a robust DevSecOps pipeline by using Jenkins and various tools.

I will share my journey in 3 parts since there is too much material and reading them in a single story may be difficult and take a lot of time.

This is the part 1 if you are interested: https://medium.com/@ataseren/creating-a-devsecops-pipeline-with-jenkins-part-1-a863566b4bc3

In this 2nd part, I will talk about dependency check, SBOM generation and secret detection.

• Dependency Check: Performs a scan on dependencies of a project and reports if there are any vulnerabilities on them.
• SBOM: Software Bill Of Materials is a list of components in a piece of software. SBOMs are useful to determine used technologies in a project and can be used for additional security scans.
• Secrets Detection: Can prevent accidental code-commit containing a secret.

Dependency Check

Dependency checks involve evaluating the external components, like libraries and frameworks, used in software development to identify security vulnerabilities, ensure compliance with policies, and mitigate risks.

For this purpose, I used OWASP Dependency-Check (ODC). It is an open-source Software Composition Analysis (SCA) tool that attempts to detect publicly disclosed vulnerabilities contained within a project’s dependencies. You can learn more from here.

First of all, you must install the Jenkins plugin for ODC. This plugin will help us to perform a dependency check on our project with minimal configurations. From the plugin page that is shown above, you can simply search for ‘dependency-check’ and install the plugin.

After installing the plugin, we must choose an installation of ODC to be run in Jenkins. To choose this, go to Manage Jenkins > Tools and in this page, find “Dependency-Check installations” part. In this part, you have the option to add an ODC installation. As you can see on the image, I chose “Install automatically” instead of giving a path to an installation since it is much easier and you can update it easily by changing the version from this part. On “Add installer” drop-down menu, choose “Install from github.com” option, choose a version, give a name to the installation and save the changes. Don’t forget this name, we will use it.

Finally, to run this step, you should modify the pipeline script. You can simply add this stage to your script:

stage('Dependency-Check') {
 steps {
     dependencyCheck additionalArguments: '', odcInstallation: 'dep-check-auto'
        }
}

Here is an important note. “dependencyCheck” option under Snippet Generator can give you a script. However, it is incomplete. It just gives `dependencyCheck additionalArguments: ‘’` part. Therefore, you should do the steps and use the script above.

In addition to this command that invokes ODC, I suggest using another command, “dependencyCheckPublisher pattern: ‘’ ”. “dependencyCheck” command only invokes ODC and generate an XML file which is usually very large, hard to read and includes unnecessary details. By using this command, you can see the result on a UI that allows user to use the results much easier. You can access to that UI in the build:

Instead of having the result as a UI in Jenkins, you can directly read the file generated by ODC. It is located in the pipeline’s workspace which is /var/lib/jenkins/workspace/\<name of the pipeline\>. However, generating the result and storing in the pipeline’s workspace can be inefficient since it is hard to access and read through the workspace’s path and generated files can affect the project files or processes of other tools running in the pipeline. To avoid this, I use a Jenkins feature called “archiveArtifacts”. This Jenkins command will find the desired file(s) and give us the option to download it. Then, it is safe to remove the generated file from the workspace.
Here is a simple script of archiveArtifacts and a shell command to generate a download link and remove the file from workspace:

archiveArtifacts allowEmptyArchive: true, artifacts: 'dependency-check-report.xml', fingerprint: true, followSymlinks: false, onlyIfSuccessful: true
sh ' rm -rf dependency-check-report.xml*'

In the archiveArtifacts script ‘allowEmptyArchive: true’ means that this step does not fail build if archiving returns nothing. artifacts: ‘dependency-check-report.xml’ is the aprt which we choose the file. Wildcards can be used in the name of the file too. ‘fingerprint: true’ means that Jenkins fingerprints all archived artifacts. ‘onlyIfSuccessful: true’ means that Jenkins archives artifacts only if build is successful. Finally ‘followSymlinks: false’ means that all symbolic links found in the workspace will be ignored.

pipeline {
    agent any

    stages {
        stage('Checkout') {
            steps {
                git 'https://github.com/ScaleSec/vulnado.git'
            }
        }
        stage('Build') {
            steps {
                sh 'mvn clean package'
            }
        }
        stage('SonarQube Analysis') {
            steps{
                withSonarQubeEnv(installationName: 'sonar-local') {
                  sh "mvn clean verify sonar:sonar -Dsonar.projectKey=vulnado -Dsonar.projectName='vulnado'"
                }
            }
        }
        stage('Dependency-Check') {
   steps {
       dependencyCheck additionalArguments: '', odcInstallation: 'dep-check-auto'
       dependencyCheckPublisher pattern: ''
       archiveArtifacts allowEmptyArchive: true, artifacts: 'dependency-check-report.xml', fingerprint: true, followSymlinks: false, onlyIfSuccessful: true
sh ' rm -rf dependency-check-report.xml*'
   }
  }
    }
}

And this is the result of the build:

In this picture, you can see the small download icon next to the build, artifacts listed above and a graph generated by ODC.

Here is the ODC UI result:

You can find the information about invocation of ODC and the path of generated XML file on console output. It is usually at `/var/lib/jenkins/workspace/vulnado/./dependency-check-report.xml`

Here is another note about your first pipeline run with this step. Depending on you download speed, first run with this step will take relatively longer time than you future runs because at the first run, ODC downloads vulnerabilities from NVD database. You can speed it by using a NVD API key but it is unnecessary since this is a one time process.

Here is another note about your first pipeline run with this step. Depending on you download speed, first run with this step will take relatively longer time than you future runs because at the first run, ODC downloads vulnerabilities from NVD database. You can speed it by using a NVD API key but it is unnecessary since this is a one time process.

SBOM Generation

SBOM stands for Software Bill of Materials. It’s a list of all the components, libraries, and dependencies that are used in building a software product.

SBOM enhances transparency in the software supply chain, allowing stakeholders to understand component composition and origins. This transparency aids in effective risk management by identifying and prioritizing security risks. SBOM also ensures compliance with regulatory standards.

Also, SBOM facilitates efficient vulnerability management, helping teams quickly address security issues. In the event of a security incident, having an SBOM accelerates response efforts, and its integration into the DevSecOps pipeline enables continuous monitoring for ongoing security assessment throughout the software development lifecycle.

For this part, we will use Syft tool, which is a CLI tool and Go library for generating a Software Bill of Materials (SBOM) from container images and filesystems. Before doing anything on Jenkins side, please install Syft by simply using this script:

curl -sSfL https://raw.githubusercontent.com/anchore/syft/main/install.sh | sh -s -- -b /usr/local/bin

You can find additional installing methods, guides and more details about Syft on GitHub page of Syft.

Now, we can run Syft on Jenkins by using a shell command. When you type `syft --help` on your terminal, you will see various ways to run Syft for various cases. In our case, we want to scan a directory, which is the directory of Vulnado project. `syft scan dir:path/to/yourproject ` is the command for our case. If you inspect the file hierarchy of Jenkins (you can run a command such as `pwd` in a pipeline or manually inspect the Jenkins files and directories), you will see that everything that we do in the pipeline happens in /var/lib/jenkins/workspace/<name of the pipeline>. When we fetch the project files, with our “Checkout” stage, the project is located in this path. Since we are at the same path as the project, we can simply enter “.”(dot) as the path to be scanned.

Other than this, we should add `--output` flag to our command to determine the format of the SBOM and its location. You can use this flag like this: ` --output <format>=<file>`. To sum up, we should add this command in our stage: `syft scan dir:. --output cyclonedx-json=sbom.json`
I chose “cyclonedx-json” format because CycloneDX is a very common SBOM format and its JSON version is easier to read than default one. You can change it if you want.

In addition to this command, I will add the artifact commands like previous stage. Here is the final code after this step:

pipeline {
    agent any

    stages {
        stage('Checkout') {
            steps {
                git 'https://github.com/ScaleSec/vulnado.git'
            }
        }
        stage('Build') {
            steps {
                sh 'mvn clean package'
            }
        }
        stage('SonarQube Analysis') {
            steps{
                withSonarQubeEnv(installationName: 'sonar-docker') {
                  sh "mvn clean verify sonar:sonar -Dsonar.projectKey=vulnado -Dsonar.projectName='vulnado'"
                }
            }
        }
        stage('Dependency-Check') {
   steps {
       dependencyCheck additionalArguments: '', odcInstallation: 'dep-check-auto'
       dependencyCheckPublisher pattern: ''
       archiveArtifacts allowEmptyArchive: true, artifacts: 'dependency-check-report.xml', fingerprint: true, followSymlinks: false, onlyIfSuccessful: true
       sh ' rm -rf dependency-check-report.xml*'
   }
  }
  stage('Generate SBOM') {
            steps {
                sh '''
                syft scan dir:. --output cyclonedx-json=sbom.json
                '''
                archiveArtifacts allowEmptyArchive: true, artifacts: 'sbom*', fingerprint: true, followSymlinks: false, onlyIfSuccessful: true
                sh ' rm -rf sbom*'
            }
            
        }

    }
}

And this is the result of the build:

Note: Artifacts may be not visible at the end of the build. Refresh the page and you will see both the artifact list and the download button next to the build timeline.

Secrets Detection

Secrets detection is a critical aspect of DevSecOps that involves identifying and managing sensitive information, such as API keys, passwords, and security tokens, embedded within the codebase. By implementing secrets detection early in the CI/CD pipeline, organizations can prevent unauthorized access, mitigate security risks, and ensure compliance with industry standards.

For this part, we will use detect-secrets, an aptly named module for detecting secrets within a code base. This is a simple but highly effective tool. It is very customizable for various purposes. To show you how to start using it in a pipeline, I’ll use its basic features but I encourage you to look for other capabilities of this tool.

This tool is written in Python and installed with Pip. Because of this, we need an adjustment on Jenkins to use it in our pipelines. First of all, let’s install the tool with the command:

pip install detect-secrets

After the installation, you may need to add the path of this tool to Jenkins. By default, Pip installs this and other tools into $HOME/.local/bin. In my case, Pip installed detect-secrets into /var/lib/jenkins/.local/bin. This may vary according to your Jenkins and Pip configuration. To make sure that the tool is available in the pipeline, we must add this location to the path.

You can do this manually by using jenkins user and changing the path on the terminal. Alternatively, here is an easier way to do it on Jenkins UI. From the dashboard, go to Manage Jenkins > System > Global properties. In this part, check the Environment Variables and add a variable. For the name, type “PATH+WHATEVERYOUWANT”. It must be all caps and no space. I did this because sometimes I had issues with other naming methods. For the value, add the path of your tool.

After these steps, detect-secrets is ready to use. I added this stage to my Jenkinsfile. I made it write the output to a text file and give it as an artifact, like previous stages.

stage('Secrets Detection') {
            steps {
                sh 'detect-secrets scan > secrets.txt'
                archiveArtifacts allowEmptyArchive: true, artifacts: 'secrets.txt', fingerprint: true, followSymlinks: false, onlyIfSuccessful: true
                sh ' rm -rf secrets.txt'

            }
        }

A small tip: By using `all-files`, you can scan all files recursively, as compared to only scanning Git tracked files. This option is very useful if you are not using Git in your project or if you are using another VCS.

Here is the final code:

pipeline {
    agent any

    stages {
        stage('Checkout') {
            steps {
                git 'https://github.com/ScaleSec/vulnado.git'
            }
        }
        stage('Build') {
            steps {
                sh 'mvn clean package'
            }
        }
        stage('SonarQube Analysis') {
            steps{
                withSonarQubeEnv(installationName: 'sonar-docker') {
                  sh "mvn clean verify sonar:sonar -Dsonar.projectKey=vulnado -Dsonar.projectName='vulnado'"
                }
            }
        }
        stage('Dependency-Check') {
   steps {
       dependencyCheck additionalArguments: '', odcInstallation: 'dep-check-auto'
       dependencyCheckPublisher pattern: ''
       archiveArtifacts allowEmptyArchive: true, artifacts: 'dependency-check-report.xml', fingerprint: true, followSymlinks: false, onlyIfSuccessful: true
       sh ' rm -rf dependency-check-report.xml*'
   }
  }
  stage('Generate SBOM') {
            steps {
                sh '''
                syft scan dir:. --output cyclonedx-json=sbom.json
                '''
                archiveArtifacts allowEmptyArchive: true, artifacts: 'sbom*', fingerprint: true, followSymlinks: false, onlyIfSuccessful: true
                sh ' rm -rf sbom*'
            }
            
        }
        stage('Secrets Detection') {
            steps {
                sh 'detect-secrets scan > secrets.txt'
                archiveArtifacts allowEmptyArchive: true, artifacts: 'secrets.txt', fingerprint: true, followSymlinks: false, onlyIfSuccessful: true
                sh ' rm -rf secrets.txt'

            }
        }

    }
}

And this is the result of the build:

After a long break, this is the end of Part 2. I will complete and upload Part 3 ASAP. I’d love to hear your comments about the format and content of this story, so I can improve them.

Thanks for reading. I hope it helps!