Trigger Jenkins Job from BitBucket with a Web API Middleware

A Use Case

As a DevOps guy at my company, I was working on Jenkins to automate some work-flows of Node applications. We use BitBucket as SCM and Jenkins as a CI\CD Server where BitBucket notifies the Jenkins server via a Webhook whenever a change occurs in SCM. We use pipeline jobs where we write the script once and put the Jenkinsfile into the SCM.

An ideal Jenkins work-flow that we were expecting at that time was such that when a dev guy push some changes to any feature branch, Jenkins should clone the repository against that feature branch, execute the steps defined in the pipeline and optionally notify developers if something goes wrong or deploy the code directly to the server.

We, then, came across at a point that the hard-coded groovy script that we used in the pipeline only clones the repository from a specific git branch regardless of where ever the code is pushed in the SCM. That was really challenging as we were utilizing the same work-flow against our open-source projects with Travis-CI. One thing that we thought was to access the BitBucket Webhook payload in the pipeline anyway but we came to know that there was no built-in way to get the Webhook JSON payload received from Bitbucket in the pipeline jobs.

I did take a look at the Jenkins BitBucket Plugin but that works well with a Free-Style Job not with a Pipeline Job according to this issue.

The Solution

A solution to the problem is to use a Middleware Application between BitBucket and Jenkins server. This middleware application, in a nutshell, will listen from the BitBucket, if BitBucket gets some latest code it will trigger the Webhook to the middleware application including the JSON Webhook payload containing the meta-data such as branch name, commit, author etc. This middleware application will then parse the Webhook payload, extract necessary attributes and pass the required arguments to the Jenkins. Jenkins will receive the required parameters via a Parameterized Jenkins Job from the middleware which we will eventually get in the pipeline job script. The case is illustrated below:

Middleware Application

I have prepared an ASP.NET Core 2.0 Web API middleware which listens for BitBucket Webhook, get & parse the Webhook payload, extract the Branch Name and Repository Name and forwards the Branch Name to the parameterized Jenkins job as a HTTP POST request. You can get the application directly from GitHub or you can run the application as a Docker Container and pass the Jenkins address with a JENKINS_ADDR environment variable flag to the container.

Let’s see a step-by-step guide to setup our own such CI work-flow next.

Example Work-flow for Jenkins & BitBucket

Getting the middleware

I’ll use the Docker Container for the middleware by downloading the Docker image from Docker Hub with the command docker image pull kjanshair/bitbucket-jenkins-trigger. You can also run the middleware directly with dotnet CLI.

Starting the middleware

I’m on a Ubuntu Server where Jenkins is installed and running at port 8080. I SSH into the machine and run the middleware as a Docker container with the command:

docker container run -d --network=host --name hook -p 5000:5000 -e JENKINS_ADDR=localhost kjanshair/bitbucket-jenkins-trigger

This command will start the container and pass the Jenkins address as the environment variable to the middleware container. Since we are on the same machine where Jenkins is installed, we will use localhost here.

It is important to use the --network=host flag in this example. If you don’t specify this flag the container won’t recognize localhost as the current running Jenkins instance. You, then, have to pass the public DNS\IP address to the container.

Jenkins Pipeline Job Configuration

Go to your Jenkins Job configuration, in the General section, select option This project is parameterised, add a string parameter named branch and in the Build Triggers section make sure that you provide a random string-based token by selecting the Trigger builds remotely (e.g., from scripts) option as this token will be used in the BitBucket Webhook as a query string parameter to identify the job uniquely.

In the pipeline script, write the groovy script for your repository URL (I’ve used my own here) as:

node {
stage ('Get Code') {
def bname = "${params.branch}"
git branch: bname, url: ''

Now our Jenkins side is ready to listen from the middleware. Let’s configure the BitBucket to POST Webhook to the middleware instead of posting directly to Jenkins.

BitBucket Webhook Configuration

In BitBucket, go to:

Repository Settings => Webhooks => Add webhook

Give Webhook a title and set the URL to point to the middleware with the syntax:


It is important to know that the middleware currently handles only PUSH triggers.

That’s all we have to do at the BitBucket side. Now finally try to push some code to the repository and take a look at the logs of the middleware with the command docker container logs -f hook which will display the Branch Name and the Token that it received from BitBucket upon receiving the Webhook and will trigger and pass the branch name to the Jenkins Parameterized Job. That’s All! Happy DevOps. :)