Lessons learned after two years of ATDD

Introduction

The practice of ATDD (Acceptance test–driven development) comes from the necessity of having a concise and unambiguous way of communicating, with examples, how something have to work, so two parties (in this regard, product owners (POs) and the dev team (Devs)) use the same language.

Lower level type of testing like Unit Tests or Integration Tests that are of a more technical scope are also used.

Adopting ATDD adds a safe net of tests that will attempt to catch changes that could break the application because you are testing it as a black box where you have inputs and outputs. This tests are born from an specification comming from the POs and an implementation comming from the Devs.

We will be concerning mostly with Cucumber and Gherkin (specification langage for Cucumber). If you haven’t checked it yet, go ahead and have a look.

Now I will try to explain some of the pitfalls and good practices that I’ve found over the past 2 years.

Pitfalls

When someone first aproaches ATDD might be misled to think that it will be easy or that it will solve all the testing problems of the application in development. I’m afraid you will be disappointed by the following lines, but trust me… it’s well worth.

It’s easy

ATDD it’s not easy, PERIOD. It requires a lot of discipline and communication to make it worthwhile. This communication should be bi-directional, from the POs to the Devs and the other way around. But in the end what you get is an easy to read living documentation reflecting how your systems behaves (as an agreement between POs and Devs), instead of a 250 pages long document with the requirements someone else though would be necesary.

I will never fail again!

Even with a lot ot tests, things can go wrong. Bugs might crop up, unexpected behaviors might happen. We are human after all. When this happens, add a test to check this new bug and fix it in your code. Your safe net of test is as safe as you make it. A flimsy net will let bugs get through, a strong net it’s more likely to catch something that breaks or that have been broken in the past so it won’t happen again.

This will solve all my problems

Adopting this practice will not solve any problem as it is. You need to devote time and space to master it, and it will enhace the team’s communication skills. After all, the main porpuse of ATDD is having a concise way to explain the Devs what the business side wants.

Good practices

Not every thing is bad, otherwise I wouldn’t be here. It’s also rewarding to have the knowledge that your application is less likely to break (again?) from past problems.

Now I will try to outline some practices that I’ve found useful on the past.

Avoid coding tests and application at the same time

When programming, first do the tests for the feature, then the code to pass the tests.

When refactoring avoid modifying the test’s code AND the application code. If you modify both at the same time you will never be sure if you have broken any of it.

Prefer simple features with few and specific scenarios

Simpler scenarios are easier to understand and leave less to interpretation. That your application is complex does not mean that your features and scenarios have to be complex too, after all you don’t want to have to make test out of your own test code.

A concise example is the best way to comunicate to the Devs what you want.

Avoid dependencies whenever possible

Dependencies between 2 or more steps is better avoided. Steps should be as stand alone as possible, trying to avoid tieing the execution of 2 or more together.

If your steps have no dependencies they can be easily refactored.

Shared context

It’s better to share no data among steps, but sometimes it’s not easy or makes testing more complicated.

A concept that I borrowed from CucumberJS (World concept) and that I’ve used on most languages I’ve programmed on is to have some shared object (call it SharedContext, World, whatever) where steps can put and get data that is required by some other step or share some functionality.

This conflicts with my previous statement, but as I said, it’s not always easy to do so.

Reuse steps

A way to reuse steps is to avoid starting anew on each and every scenario. If you already have an step doing what you need, reuse it. It will save time to you and the Devs.

Steps are interchangeable

Given, When and Then steps, from an implementation point of view, are interchangeable. You can fit any of them in the other positions. The distinction between them it’s to clarify their porpuse.

Backgrounds

You can use Backgrounds to summarize steps that are repeated throught several Scenarios in a Feature. That way you only specify your preconditions once for the whole Feature. For example:

Summary steps

If backgrounds get to verbose or too long, you can extract some of the functionalty and write it down as a single step. This might make the implementation of that particular step a bit longer, but the Scenarios should end up being simpler and shorter. For example, we could write this group of steps:

could be rewritten as a summary like this:

This might seem trivial as examples go, but the resulting Scenario will be a bit more concise

Verbal tenses matter

Given, When and Then should reflect past, present and future.

  • Given steps are the past, some pre-condition that must be met before we start.
  • When steps are the present, what we are doing.
  • Then steps are the future, what will happen.

This is more of a convention than anything else.

Regular expressions

From the Dev point of view, knowing your regular expressions will help you a lot. As an example, an step that could be used as Given or When could be implemented like this:

This is a very trivial example, but ilustrates my point. An article that I usualy check for this matter is Just Enough Regular Expressions for Cucumber. It’s a bit old, but still works today.

Repeatable runs

Whenever possible, treat identifiers or names used on the scenarios as an alias, and not as the final identifier. Save both, the alias and the identifier in the shared context so it’s easily accesible from other steps. Something like this

This way, you always have a fresh identifier to repeat the test without having to restart your application or delete anything from the database.

Conclusion

Although getting to master ATDD might seem a daunting task, the benefits derived from it outweight the drawbacks by far. Hope you can find something useful in this article to make the process easier.

Not only you are getting away with a more robust application, you also get a living documentation with examples on how it works born from the communication with all the parts in your team.

Some books worth reading are Specification by Example and The Cucumber Book