Gherkin — is it doing what you think?

Gherkin — the ubiquitous Given — When — Then language — used in automated tests everywhere. It’s designed to provide stakeholders with human-readable tests to provide confidence that software created meets the criteria. But there’s a glaring problem with it that has largely gone unnoticed…

Let’s deliver value by writing Gherkin

At my organisation we use Gherkin a lot. It’s the starting point for every automated test (that’s a story for another time 😩) and a lot of time is spent on crafting it. A typical process goes something like this:

1. A user story is refined with a set of scenarios. The team (or more usually the tester) spends (a lot of) time crafting Gherkin for these.
2. The Gherkin is then added to feature files and a set of tests is created in the code base
3. Tests are run, pass and the code submitted for code review
4. The code is deemed good and is submitted to the central code repo
5. High-fives all round on a job well done!

So everything is fine with this, we are delivering excellent automated acceptance tests to provide confidence for our stakeholders. 👍

Until one day this happened..

The stakeholders

I was having a discussion with a project stakeholder about Test Summary Reports (I know, and we are “Agile” 😢) and they said

“Of course, I have no idea whether what is being developed is what we need”

Whoa there! The Gherkin! The amazing non-technical, business-readable tests we spend hours lovingly crafting. One read of the gherkin will provide all the information you need! Why aren’t you reading the Gherkin!

And then it struck me.. how can they? Step 4 above, committing to a central repo, a repo tucked away on a server in the cloud, locked down with permissions on who can access it. The stakeholder has no way of ever seeing the Gherkin..!

So the fundamental reason for writing Gherkin — to provide confidence back to the business — was being lost. It was a body-blow.

But as ever, there is a solution..

Enter Pickles

There exists numerous tools that aim to “release” Gherkin from the depths of a repo, but one free tool stood out and that was Pickles.

Link: http://www.picklesdoc.com

This tool scans a project for feature files and then exports the content into HTML. This can then be exported to a file share or static website, and this HTML can then be shared with anyone.

The process we use is to implement is:

1. Create a central location to store the exported HTML
2. Add the Pickles package to the project (we predominantly have .NET projects)
3. Update the “commit to master” pipeline to include Pickles steps to generate the HTML and then export it
4. Share the location from Step 1 with the stakeholders for that project

It’s as simple as that. In 30 minutes we had exposed the entire Gherkin for our first project and placed it in the hands of the stakeholders.

Our happy place

We are very excited to have discovered Pickles and will be rolling it out to all our products. We have created a static site on Azure and will store all the HTML output here, hand-crafting an index page to list all the products.

The benefits to us are:

• Stakeholders have are able to read the acceptance tests and (one hopes) gain confidence that what is being developed is what is needed
• Gherkin is now delivering on it’s sole objective!
• The time and effort generating Gherkin becomes worthwhile (assuming of course these are acceptance tests and do need Gherkin — that’s for another blog!)
• And interestingly, we have a single place that contains the Gherkin from all teams in the organisation, allowing us to review its use and therefore see how we can help teams to improve.

I would recommend anyone using Gherkin to consider whether it is delivering on it’s objective and if not, consider implementing a tool like Pickles.