Cucumber @ ING: making it part of the agile workflow

Since this year, ‘the business’ and ‘the IT’ of ING’s liquidity management live together on the same floor. Together, we set the business requirements of our systems through conversations with our Product Owner and also get review feedback from our internal business customers. This sounds all very nice on paper, but ‘business people’ and ‘IT people’ in reality can speak very different languages. As such, we looked for an approach for building a shared understanding of what should be built that bridges the gap between technical and functional understanding. This is when we started experimenting with Behaviour Driven Development (BDD) and the Cucumber framework.

What is Behaviour Driven Development (BDD)?

Behaviour Driven Development (BDD) aims to bridge between business analysts and development teams by providing a common language which is purely based on business requirements. What separates BDD from its predecessor TDD (Test-Driven Development) is that tests are written as plain text scenarios that can be read and verified by the non-technical colleagues and stakeholders. In other words: no programming skills required. ‘The Business’ can actively participate in the automated test cases review process and give their feedback to refine them.

What is Cucumber?

Cucumber is an open source tool that supports Behaviour Driven Development (BDD) by offering a language, called Gherkin, to write tests in a way that both non-technical and technical colleagues understand. They collaboratively describe the intended behaviour of the system (acceptance criteria) in this special language. The Gherkin language is essentially normal English (or any of the other supported languages) extended by a set of special keywords to give structure to enable automated testing. Let’s look a simple example from our favourite domain (yes, banking).

Example of a Gherkin feature

As you can see, Gherkin reads like normal English and the described acceptance criteria are understandable by anyone who, in this case, knows what a bank account is. Each sentence in Gherkin always starts with a keyword (coloured in yellow), except if it’s a comment. The most important keywords are:

• Feature defines the logical test functionality you will test. Each Gherkin file begins with this keyword.
• Scenario describes a specific example that is related to this feature. Scenarios are like individual tests in other testing frameworks and can either pass or fail.
• Step keywords (Given, When, Then, And, and But) are used to write down the scenario by first describing the current situation or start state (Given), followed by an occurring behaviour (When). Finally, you give the expected result or end state (Then). More complex clauses are formed by using And and Or.

You may think now: “okay, so these tests are defined in a friendly way, but how can I run these tests?”. Very valid question. To make our tests actually runnable we define so-called step definitions, which are Java methods that ‘implement’ Gherkin. See the example below.

What step definitions could look like

Step definitions define a certain pattern that should match Gherkin sentences. We also use variables such as {string} and {int} to make the step definitions more reusable and so that we can use that value in the actual code. When you ask Cucumber to execute the tests, it runs through the Gherkin file line-by-line and looks for a matching step definition to execute. Observe that if we replace the Gherkin sentences by their related step definitions and concatenate it all together, we actually end up with a plain old JUnit test!

Now, we should note that Cucumber is not new to ING. 5 years ago, BDD with Cucumber was introduced with an internal framework called ‘InTRestING’, a catchy acronym that stands for Interface Testing for REst API’s within ING.
Even though, it couldn’t be more relevant for us today as our application must work with many different message formats from many different systems and handling is very much dependent on the functional meaning of messages. We need a common language to talk with the owners of those systems, which led us to BDD and Cucumber.

This article walks through a typical process that starts with a user story still residing in the Product Backlog and ends with the lovely green checkmarks of passing tests. The goal is to get others started.

Five step process to make Cucumber part of your agile workflow

One needs to follow BDD principles to get the maximum out of Cucumber, because Cucumber is not a testing tool but a BDD tool. If you don’t want to define the functional requirements collectively with stakeholders and want to do testing the old-fashioned way, then stick to your JUnit. As said before: if we replace the Gherkin sentences by their related step definitions, we actually end up with a JUnit test, so Cucumber’s value is not in technical testing capabilities. The added value of Cucumber is to have the discussion upfront.
You may now think: “okay, that all sounds nice, but also very vague”. To help you get started, we will walk you through our current process with concrete steps and examples based on our experiments. A basic outline:

1. Pick a user story from the product backlog that will be included in a sprint soon.
2. Do ‘Example Mapping’ exercise (approx. 20 min) to uncover Acceptance Criteria.
3. Convert resulting Acceptance Criteria into Gherkin.
4. Implement tests when story is picked up.
5. Periodically report to business on the test results.

After the first three steps you have agreed on the functional requirements and converted them into Gherkin. When the story is later added to the sprint in step 4, the acceptance criteria are already available to the team and, as a side-effect, the discussions during the Example Mapping also help you refine the user story. Finally, in step 5, you keep all stakeholders up to date on the progress of implementing the features.

Step 1: Pick a story

The first step is to pick a user story that you want to use. This is preferably a story that will likely be picked up in the near-future. For this article, we will use the first story that our team approached in a BDD way, which was about giving support users of the system the functionality to replay previously received messages. For the sake of this article we leave out details and complexities.

Our exemplary user story

Step 2: Example Mapping

Now that we have picked our story, it is time for the Example Mapping exercise. The exercise was introduced by Matt Wynne, who is an expert on BDD and Cucumber. His original article can be found here. The exercise should at least be attended by what one calls ‘the three amigos’: a developer, a tester and a product person. We usually also invite the team members that will most likely pick up the story, as it helps them better understand the content. The exercise has two goals:

Goal 1: agree the functional rules (requirements) of a story.

Goal 2: come up with examples of these rules.

So how does it work? The story itself is written down on a yellow card. Then, participants can take 3 types of actions:

1. Write a rule for each known acceptance criteria on a blue card under the story.
2. Write each example for a rule on a green card.
3. Write each open question on a red card on the side to discuss later.

There are no rules on who can use what kind of cards. Anyone can suggest adding something to the table and in most cases others disagree or don’t understand, which is exactly what you want as that sparks discussion among participants. For writing down the examples, Matt advises the so-called ‘Friends episode naming convention’, i.e., every example starts with ‘the one where..’. We haven’t tried that convention in practice yet, but just love the name.
After the 20 minutes timebox has elapsed, you assess the resulting constellation of stickies. For the sake of this article, let’s assume that the team came up with the following result:

Our hypothetical example mapping

Either you approve the result, in which case we continue to the next step, or you decide that more time is needed. In the latter case you can plan a follow-up Example Mapping. We use three simple rules of thumb introduced in Matt’s article:

1. A table covered in red (question) cards tells us that we still have a lot to learn about this story.
2. A table covered in blue (rule) cards tells us that this story is big and complicated. Perhaps we can slice it? Take another yellow (story) card and put the sliced story on the backlog.
3. A rule with many examples might be over-complex. Are there multiple rules at play that need to be separated?

In our case we have identified two business rules in blue, each with two examples in green. Also, the developer wondered whether messages can be replayed at any point in the future, as this would influence implementation. Below you see a picture during our own Example Mapping (excuse us for not using the right colours, this was an early experiment).

First experiments

Step 3: Convert into Gherkin

So now we have a bunch of coloured stickies on the table and we want to convert them into Gherkin. This last step could be done together with the business, but that time is probably better spent on functional discussions. Therefore, we recommend to assign the task to one team member and later collectively approve the Gherkin text. Converting into Gherkin might seem like quite a complex task, but actually it is mostly a simple copy-paste exercise. See the illustration below.

The relation between Example Mapping and Gherkin

The colours correspond to the colours of the stickies. You start by typing the Feature keyword, followed by the story name. We usually slightly edit the name and, for instance, skip the ‘As a user,’ part. After that, you want to document the identified business rules and the questions that were left for future reference. Don’t worry about how you format this text as, in Gherkin, any sentence that does not start with a keyword it treated as a comment. Finally, the green example stickies become the scenarios, which start with the Scenario keyword. Writing examples into scenarios is the most work in this step, as they should be written in a Given-When-Then structure, while you probably wrote the examples down as Friends episodes.
Now the only thing left to do is to add it to your project folder. When the story is picked up, the test description will conveniently be in place already.

Step 4: Picking up the story

At some point, the story is added to the sprint backlog. Our tip is to update your Definition of Done to include implementing and passing the Cucumber tests if Gherkin is written for the story. The developer creates the required step definitions in Java. Step definitions can be used in multiple features, so being consistent in your wording pays off in better reuse. We have seen an example of the step definition code earlier. When running it in an IDE like IntelliJ, running the now-automated tests shows you a window like this:

Your IntelliJ window when everything is fine

Step 5: Reporting back to the business

Of course, showing your IntelliJ screen with many green checkmarks probably won’t impress your stakeholders. Cucumber provides you with the tools to talk about acceptance criteria in a language anyone can understand. It would be great if it would also offer a way to report on the progress you made on passing these tests in an equally friendly matter. Cucumber partly does that for us. When you run Cucumber, it already generates a JSON file containing the results of the tests, including with features, scenarios and sentences passed successfully or not. It is then up to us to create some nice visuals upon that. Our personal favourite library to take care of this is called ‘cucumber-reporting’ and can be found here. Both more basic and more flashy libraries are available, but we like this one as it offers a good compromise. Refer to the GitHub page or the page of the related maven plugin for more information. Below you see an example screen from their page.

Visual reporting everyone understands

Conclusion

We introduced BDD and Cucumber only a few months ago, but the benefits are already evident. Communication and understanding between business and IT has improved and requirements are more clearly defined. This speeds up development and reduces rework. However, we also still have a long way to go. When new mindsets (BDD) and processes are introduced, the team always needs time to build maturity. There are always moments when deadlines are tight and you are tempted to revert to the old ‘testing comes later’ approach. In this article, we shared our first findings and, hopefully, get our colleagues and readers started as well.