Testing your pull requests with Jenkins

If you’re a nodeJS developer words like mocha, chai, supertest and superagent might sound familiar to you. It’s a regular practice to combine your git flow with tests so you guarantee your code is fully working and avoid some undesired regressions. This is often used in open source projects and TravisCI is a great tool for that. However, if you have private Github repositories, you’ll have to pay to use TravisCI.

On the other hand, Jenkins is an open source automation tool that allows you to perform that kind of tasks (and many, many more) for free. Jenkins can make your life easier at so many levels. I’m not diving into this with much detail because there are a lot of articles about the subject. Thus, how can Jenkins help us running our tests after a pull request is submitted?

I’m going to assume that you already have a running Jenkins instance with Github Plugin installed and configured :) If that’s not the case, do it before proceed.

  1. Install GitHub pull request builder plugin.

Inside your Jenkins instance go to Manage Jenkins > Manage Plugins > Available Tab and search for “Github pull request builder”. After the installation, if you go to Manage Jenkins > Manage Plugins > Installed you should see the plugin.

Github Pull Request Builder is on the installed plugins list

You can configure the plugin in two different ways: You can configure with a Github account that is an admin of your repository, or if you don’t own an admin account you can use a regular Github account.

“Hey Luís, What’s the difference? What’s better?!”. Great questions! Basically the difference is that with an admin account you can setup Github webhooks so Jenkins is automatically notified every time a pull request is opened. Without an admin account we have to setup a timer so Jenkins can periodically check Github for new pull requests. In this article I’m using a non-admin account.

2. Go to Manage Jenkins > Configure System > GitHub pull requests builder section.

Github Pull Request Builder Plugin Configuration

On Github Server API URL type https://api.github.com (1). Then click on Create API Token button (2). Enter a temporary username and password and click on Create Token. Refresh the page. Now you’ll notice Jenkins just filled the Shared secret field for you. Now, in the Credentials dropdown select the “auto generated token credentials” (3). Now click on Advanced (4). In the crontab line field just type * * * * *. This line will make Jenkins to check if there is a new pull request — or any change in an existing one — in every minute. We’ll take that for tutorial purpose. You can change that later. For additional plugin configurations, such as whitelist users, admin users, etc, you can check here.

3. Create a new job to run your tests.

Go to New Job > specify a new name and choose Build freestyle project (first radio box). Ok.

3. Click on the job you just created and go to Configuration, on the left bar of Jenkins.

4. Choose a project name (1). Enter your Github Project url (2). Enter your repository URL and choose your Github Credentials (3). Click on Advanced (4). In the Refspec field type “+refs/pull/*:refs/remotes/origin/pr/*”. In the Branch Specifier field type “${sha1}” (5).

Setting up your new job

Now the final touch. Scroll down until you find Build Triggers section. When you do you stop and check the GitHub Pull Request Builder (1). The GitHub API credentials dropdown should be automatically filled. If not, just select the credentials you created before. Yeah, the ones Jenkins auto generated for us :) At this point you can also setup some additional configurations like we did before in the Jenkins Configuration section. This is a great feature so you can have different configurations for different jobs.

In the Build section, we click Add build step button and then Execute shell. Now you are able to type the commands you want Jenkins to perform in the build step. In our particular case, as this is a NodeJS project, I install all the modules in the package.json file and then start an npm script to run all my tests. Done!

You now have your Jenkins instance running your tests every time a pull request is opened or modified. In those cases, you should see something like this, in the Github page.

This is just a straightforward explanation on how to get things started. Now you can customise it the way you want.

In a future post I’ll show you how to get the test results in a more pleasant way than reading them from the console output.

If you noticed any error or have any suggestion to share about this or other article just get in contact!