Test automation with Jenkins and RSpec

Testing your apps is only feasible if you enforce that all the tests pass on each release. A good way to achieve this is to have automated testing and reporting: tests that are executed after each release of the staging version.

The good thing about Jenkins that it lets you have this easily. I’ll try to make a simple guide of how to implement it. In the end, this is the workflow I want to achieve:

  • Deploy staging version
  • Run tests
  • Generate reports, display them in Jenkins, and also send them through email


Running the tests

I’m not going through all the little things about configuring Jenkins, as each system is different and it goes a little far out the scope of this guide.

First create a new blank job for the tests. In the settings of this job, go to the execute shell box, and write the following commands:

git checkout $deploy_branch
git pull origin $deploy_branch
bundle install
RAILS_ENV=test bundle exec rake db:migrate
RAILS_ENV=test bundle exec rspec spec/* — format html — out rspec_results/results.html — format RspecJunitFormatter — out rspec_results/results.xml
echo “Finished tests!”

Now RSpec will create two test reports with two different formats: a HTML for web visualization and a xml for visualization in Jenkins. The html format is built-in Rspec, so we’re good.

Displaying the reports

Now let’s display the report after the test runs. We’ll display them in two ways:

  • Set a graph in the status page
  • Send an email with a report

For displaying the graph we’ll need this plugin for Jenkins. In the “add build step” dropdown, select “Publish JUnit test result report”. Add this to the xml report path: “rspec_results/*.xml”

Now for the email. I’m going to send an email with a link to the report html file. To accomplish this we’ll use the Doclinks plugin to store the file, and the Email-Ext plugin to send the email.

Now for the document publishing we add another build step (publish documents) , and fill in the options (for the directory to archive put “rspec_results” and for the index file “results.html”).

For the email notification we need 3 things:

  • configure the email server on Jenkins
  • add an environment variable with the path of the html file to be shown in the email
  • configure the actual email

To configure the email module, head over to manage-jenkins -> Configure system menu.

Go to Extended E-mail Notification and configure your smtp server. If you just want to use a gmail email, use the following settings: smtp.gmail.com, check “use ssl”, and port 465.

And that’s it! Now for the environment variable: go to the settings of your job, and under “Build Environment”, in “Properties Content” add “HTML_REPORT_PATH=$JOB_URL$REPORT_URL”.

Finally configure the email to be sent adding an “editable email notification” under “add build step”. Set the content-type as html and the body of the message as:

Beta tests results! <br></br>
Click link for the report! <br></br>
<a href=$HTML_REPORT_PATH>link</a> <br>

Finally let’s make this job run immediately after we deploy a staging version. For that go to “build triggers” and check “build after other projects are built”, and select your deploy staging job.

That’s it! Now you have an automated testing suite with reports! How fancy!

Note: this is DevOps, so expect to have a million different problems along the way! but don’t back down, you’ll make it!