Better testing with Cucumber
TLDR: For a recent project, Jigsaw XYZ started using Cucumber and Gherkin to allow us to write more explanatory Feature Tests without repetition. It’s working really nicely so we wanted to share some of the thinking.
Wait, Cucumber… what?
Before you go any further, this post assumes you know the basics of Cucumber testing. If you don’t, have a look at their docs, here.
Got it, now what are you talking about?
At Jigsaw, we’ve long designed and developed software with London School TDD principles in mind. Recently, however, we tried out the Cucumber’s Behaviour Driven testing framework because of the verbose syntax. Cucumber uses plain-text specifications with spoken-language “Gherkin” syntax, making it easy for a human to understand the steps of a test. This can get out of hand quickly though, resulting in a load of step definitions distinguishable only by a **single** word.
Because it’s focussed on Behaviour Driven Development (BDD) it’s great for Feature testing and forces us to think about the how users will interface with the software (API most commonly) and therefore the behaviour we should expect from the code. That’s important because the same principles that shape our code should also shape our tests. In turn we can solve problems more efficiently and not get bogged down writing unnecessary code. Here’s how to use Gherkin’s keywords to keep your test steps DRY.
So you want to be more verbose but use fewer words??
Exactly.
OK, time for some examples
Here’s an example of a simple component test for an API that validates a One Time Password (OTP) generated earlier by the same service.
Let’s take a look at the behaviour of the happy path.
Scenario Outline: I can validate an OTP
Given A valid otp payload
When I call POST /otp/validate
Then POST /otp/validate should return the status code, 200Simple, right? Easy to understand? We have:
- outlined the scenario,
- used the ‘Given’ keyword to set the scene and define the payload,
- used the ‘When’ keyword to initiate the action, in this case a POST call to the API route /otp/validate, and
- used the ‘Then’ keyword to test the expected behaviour.
Now we just have to write the steps for each of the keywords and we’re good to go.
Given(‘A valid otp payload’, function () {
this.payload = { foo: “bar” };
});When(‘I call POST /otp/validate’, function () {
this.response = function() { // Perform the HTTP POST request to call the API on route /otp/validate }
});Then(‘POST /otp/validate should return the status code, 200’, function () {
this.response.statusCode.should.equal(200);
});
Great, I’ve completed Cucumber now… I guess I don’t need to read on?
Not quite. We’ve made a genuine feature test there (albeit with admittedly limited scope) and that’s all well and good for this one scenario. But if we take a look at another scenario, we’ll start to see repetition creeping in… (_dramatic music plays in the background_)
…Scenario Outline: Once validated, I can register a User
Given A valid user registration payload
When I call POST /users/register
And POST /users/register should return the status code, 201
and the steps:
…Given(‘A valid user registration payload’, function () {
this.payload = { bar: “baz” };
});When(‘I call POST /users/register’, function () {
this.response = function() { // Perform the HTTP POST request to call the API on route /users/register }
});Then(‘POST /users/register should return the status code, 201’, function () {
this.response.statusCode.should.equal(201);
});
Hopefully you’ll see that it’s largely identical to the earlier step definitions! In fact, the only things that differ are the payload type, API route i.e. /otp/validate vs /users/register and the expected status code i.e. 200 vs 201.
Despite changing only 18 characters, we’ve written an extra 9 lines of code!
So if, like me, this brings you out in a cold sweat, you’ll be asking, “What can we do to stop this madness before it spins out of control?”
Time to interpolate!
The first thing we can do is interpolate (or substitute) some strings. We’ll use Gherkin’s {string} keyword to help with this.
Take the line When I call POST /otp/validate and let’s interpolate the path. In doing so we can reuse the same ‘When’ step definition for both scenarios. It looks like this:
When I call POST “/otp/validate”
// Or,
When I call POST “/users/register”// And the single step definition becomes
When(‘I call POST {string}’, function (string) {
this.response = function() { // Perform the HTTP POST request to call the API on route ${string} }
});
Can I interpolate multiple times?
Sure! We could also do the same with the HTTP verb. Let’s imagine you’re now calling GET /users:
When I call “POST” “/otp/validate”
// Or,
When I call “GET” “/otp/validate”
// And the single step definition becomes
When(‘I call {string} {string}’, function (string, string1) {
this.response = function() { // Perform the HTTP ${string} request to call the API on route ${string1} }
});We might earlier have written this When step three times before and now it’s just written once. Nice, right?!
What about numbers?
So you’ve been putting your string interpolation in to practice and you’ve just arrived at the `status code`. How do you interpolate an integer into the Scenario description? Here goes:
…
Then “POST” “/otp/validate” should return the status code, <statusCode>// Now you can use the handy little Examples table to tell Cucumber what interpolate!Examples:
| statusCode |
| 200 |// And that means you can still use interpolated step definitions, this time with the ‘int’ keywordThen(‘{string} {string1} should return the status code, {int}’, function (string, string1, int) {
this.response.statusCode.should.equal(int);
});
We might earlier have had to write this definition for each route (/otp/validate and /users/register) and each verb (POST and GET) — so now we’re saving ourselves 4 times the work. That makes me happy.
Taking it even further (and perhaps too far)
I could now go crazy and start combining the two…
…
Given A valid <payload> payload
When I call <method> <route>
Then <method> <route> should return the status code, <statusCode>Examples:
| payload | method | route | statusCode |
| “otp” | POST | “/otp/validate” | 200 |
| “userRegistration” | GET | “/users/register” | 209 |// StepsGiven(‘A valid {string} payload’, function (string) {
const payloads = { otp: { foo: “bar” }, userRegistration: { bar: ‘baz’ } };
return payloads[string]
});When(‘I call {string} {string}’, function (string, string1) {
this.response = function() { // Perform the HTTP ${string} request to call the API on route ${string1} }
});Then(‘{string} {string1} should return the status code, {int}’, function (string, string1, int) {
this.response.statusCode.should.equal(int);
});
Here Be Dragons!
Use caution with this last approach. Combining Scenarios can start to become more difficult to read and understand. Remember the goal is to be more explanatory to the next developer. It’s a balance between being concise and being explicit.
Also beware of differences in the set up (usually what you’ve written in the Given’s) for each Scenario. If the expected behaviour is different, it’s probably because you are invoking that behaviour in a different way. If not, there’s probably more to be done on the design 😉
That’s it
I’ve also amended the Given step definitions here as they too can be refactored in the same way.
Otherwise, that’s it. We’re using Cucumber and Gherkin at Jigsaw.xyz for our Feature tests. It allows us to be really explicit with our Expected Behaviour Scenarios within automated tests. We’re applying the BDD and DRY principles where we can and we’re more efficient with time and effort as a result.
—
Cheeky footnote
Gherkin doesn’t actually like a string containing multiple path definitions e.g. ‘some/path/here’. I think it gets confused by the multiple /’s. That’s actually how I found out I could interpolate them as a string and refactor my steps. I probably could also have read the docs more closely… but who does that?
