Running Advanced Git Commands in a Declarative Multibranch Jenkinsfile

Jenkins pipelines have advanced since its inception and Jenkins 2.x release. Now, rather than writing verbose scripted pipelines, you can write it using their much easier and streamlined declarative syntax. You don’t have to add certain steps like checking out a branch since it is now automatically done in the declarative syntax.

Jenkins will limit what is fetched by git for performance reasons and modify the configuration for the pipeline run. This can be a problem if you want to write more complex scripts that involve your mainline branch (i.e. master) or any other branch that isn’t set in the configuration.

We want to utilize the git diff command to figure out if there were changes to certain files within a repository. The use case here is that we want to make sure that an engineer updates the CHANGELOG.md file if there is a change in the src directory of the project in a branch. This automation allows us to help keep the CHANGELOG.md file updated and accurate before merging it into the master branch. For this, we will use the following command:

git diff --name-only origin/master..origin/branch-to-compare

That will output a list of files that have been changed between the branch you are working on and the master branch.

So, in the context of a Jenkins pipeline, it would look like this:

#!groovy
pipeline {
    stages {
        stage('Check for CHANGELOG update') {
            when { expression { env.BRANCH_NAME != 'master' } }
            steps {
                script {
                    // Replace this with your SSH creds for git
                    sshagent(['CREDENTIAL_NAME']) {
                        List<String> sourceChanged = sh(returnStdout: true, script: "git diff --name-only origin/master..origin/${env.BRANCH_NAME}").split()
                        def isSourceChanged = false
                        def isChangelogUpdated = false
                        for (int i = 0; i < sourceChanged.size(); i++) {
                            if (sourceChanged[i].contains("src")) {
                                isSourceChanged = true
                            }
                            if (sourceChanged[i].contains("CHANGELOG")) {
                                isChangelogUpdated = true
                            }
                        }
                        // Changelog not updated after changing source, fail build and notify
                        if (isSourceChanged && !isChangelogUpdated) {
                            error("Source files changed but CHANGELOG was not updated!")
                        }
                    }
                }
            }
        }   
    }    
}

When you run it, you get this error:

error: pathspec 'origin/master' did not match any file(s) known to git.

It can’t get the master branch. Maybe we should try adding a git fetch before running the command so it can pull the references down with git fetch --no-tags. The reason I added --no-tags is to avoid getting all the git tags and just grab branches. But, this doesn’t pull down any other branches that are definitely available on the origin remote.

If you run it locally, you will probably pull down other branches from your remote git repository. But, Jenkins changes the git configuration when running a checkout to limit the scope of the git changes. Take this example output from a job run:

> git fetch --no-tags --progress git@github.com:Rocketmiles/REPO.git +refs/heads/BRANCH:refs/remotes/origin/BRANCH
 > git config remote.origin.url git@github.com:Rocketmiles/REPO.git # timeout=10
 > git config --add remote.origin.fetch +refs/heads/BRANCH:refs/remotes/origin/BRANCH # timeout=10
 > git config remote.origin.url git@github.com:Rocketmiles/REPO.git # timeout=10

What this means is that when you run a git fetch it is limiting the scope to just the BRANCH that is being used for the pipeline run. This is a problem since you need to fetch the master branch to do the comparison.

To get around this, we can add an additional reference to the git config so it can pull down the mainline branch with the following:

sh "git config --add remote.origin.fetch +refs/heads/master:refs/remotes/origin/master"
sh "git fetch --no-tags"

Now we are able to get the list of files that were changed between the current running branch and master through git commands.

This is the final result:

#!groovy
pipeline {
    stages {
        stage('Check for CHANGELOG update') {
            when { expression { env.BRANCH_NAME != 'master' } }
            steps {
                script {
                    sshagent(['CREDENTIAL_NAME']) {
                        sh "git config --add remote.origin.fetch +refs/heads/master:refs/remotes/origin/master"
                        sh "git fetch --no-tags"
                        List<String> sourceChanged = sh(returnStdout: true, script: "git diff --name-only origin/master..origin/${env.BRANCH_NAME}").split()
                        def isSourceChanged = false
                        def isChangelogUpdated = false
                        for (int i = 0; i < sourceChanged.size(); i++) {
                            if (sourceChanged[i].contains("src")) {
                                isSourceChanged = true
                            }
                            if (sourceChanged[i].contains("CHANGELOG")) {
                                isChangelogUpdated = true
                            }
                        }
                        // Changelog not updated after changing source, fail build and notify
                        if (isSourceChanged && !isChangelogUpdated) {
                            error("Source files changed but CHANGELOG was not updated!")
                        }
                    }
                }
            }
            post {
                failure {
                   // post to Slack
                }
            }
        }   
    }    
}

This will allow us to notify developers to update the CHANGELOG.md file if actual source is changed! But, you can utilize the changes required here to do more complex tasks with git.

Although it is a small change that Jenkins does, it can affect what you are trying to script. So, if you are looking to utilize git commands against different branches other than what is checked out, you will need to change the behavior of the git configuration.