Automated API testing

by Naomi Stern

Developing microservices with RESTful APIs means a large amount of testing will involve hitting endpoints and checking the results. From the tester’s point of view, this is a lot of doing the same thing over and over, hitting the same endpoints over and over.

As a result we concluded that automated tests was the way to go. We collectively decided to go with BDD style testing to make it easy for anyone to understand the test output.

However things aren’t always plain sailing.. Here are some steps that often go wrong:

• The tests take ages to run as the test suite grows, so people give up running them
• Writing new tests is complicated and takes long, so people don’t write new tests
• Updating existing tests when the code changes is hard so people dont do it, either ignoring the failing tests, or deleting them.

To avoid these common slip-ups we wanted to choose a framework that would make tests fast to run, easy to write new tests, and easy to update. The obvious choice would be to choose a framework made for testing Java. We are after all Java people, and that’s what we do!

There are some standard BDD frameworks for testing Java, such as Cucumber, Fitneese, JBehave, which all follow the BDD Given, When, Then style. With tools like Cucumber, and Jbehave, descriptions of the expected behaviour (sometimes called the features) is separated from the tests executable code so updating existing tests means every change needs to be replicated in two places, which is more work than necessary and, generally, a pain. This is often the reason why existing tests aren’t maintained.

We wanted tests that would be fast, easy to write and maintain. So, instead of sticking to the Java just because ‘that’s what we do’, we went with what was best for our case, and decided we would write our tests in Node.js, choosing whatever node frameworks were most suitable.

For the actual API testing we chose Chakram. This does API calls and uses Chai to make assertions on the results. We settled on this framework because it is simple to use and doesn’t include any needless extras..

An example of a call with an assertion in chakram:

Chakram uses mochas Describe, it style, and we wanted out tests to have the Given, When, Then format so we used mocha-cakes-2 which allowed us to write the BDD scenarios together with the test steps to produce a readable BDD style result.

Sample code:

Sample output:

To enable us to run the tests on multiple environments, we used config. Our tests were taking more time to run than we would have liked because we were setting up data to use for our API calls, so to save time when running the tests we used mocha-parallel-tests meaning tests can be run in parallel.

Having tests run in parallel means adding new tests doesn’t expend any more time because it’s just more things happening at the same time. Our tests are currently taking around 20 seconds to run (which is super fast considering the amount of tests we have, and the setup needed for every test!)Following this we set up Jenkins jobs that run tests during deployment. The speed at which the tests run mean it is easy to see from the output which have failed.

And that, in a nutshell, is how we built our fast, easy to maintain automated API tests.