Is BDD testing? Part 4 - Conclusions around BDD in a test strategy

Over the past few posts I’ve been exploring the behaviour driven development (BDD) process in an attempt to answer the question, is ‘BDD’ testing? I’ve done this by building a model of BDD and explored both the collaborative and outside-in development sections.

Well come on then, is ‘BDD’ testing?!

The easiest option would be to simply say no, but it’s more nuanced than that. Yes, BDD is not a solely a testing activity, it’s a process that is owned and carried out by the whole team. However, there are aspects of BDD than can help facilitate testing activities, especially during the collaborative phase. The trick is to know which aspects of BDD are beneficial to testing, and which are not.

Leveraging BDD in a test strategy

For me, a good test strategy is a rich collage of different processes, techniques and tools. The various testing activities support one another whilst also having their own unique place with Testing. BDD allows a tester to exercise activities within their test strategy, allowing them to engage with their team. A tester using BDD will be able to ask questions, collect information and help identify risks to inform other testing activities. For example:

• A tester engaged in the collaborative phase of BDD can talk to and learn from the business and development sides of the team.
• Examples can be used as a heuristic to generate test ideas. Much like a diagram or design document can be used to generate ideas.
• Testers can pair with developers during outside-in development to learn how a feature is being implemented technically to inform testing at a later date. Whilst providing feedback to the Developer.
• A good strategy will use BDD to carry out these activities but will also carry out other testing activities. Much like Developers and business owners will carry out other activities outside the BDD process.

Conclusion: Using BDD as a test strategy?

“It’s easy to lose sight of the fact that this is documentation. The whole point is executable specification, it’s documentation. And in the same way, because people think ‘oh no this is tests, more tests must be better because tests are good’, more documentation isn’t better; the right documentation is better, the right level of documentation and the right focus is better.” (Dan North, GOTO 2013 Interview with Liz Keogh & Dan North)

Unfortunately, it is currently popular for test teams to use BDD as the basis of their test strategy. There are teams ignoring the collaborative side of BDD, focusing too much on using a Gherkin syntax as means to build test cases and misunderstanding the purpose of outside-in development to focus on automating test coverage. It’s understandable why these mistakes are made. Testing has, for a long time, been too reliant on test case management-based strategies and it has led teams to make incorrect connections between Gherkin examples and test cases. Both are step-based, concerned with asserting behaviour and are written in plain language.

I’ve talked at length at conferences about how attempting to use Gherkin-based scenarios over test cases is a false economy. This results in teams following the same test case management strategy but with a different syntax. It’s important to remember that Gherkin is for development guidance, not test coverage. Accepting that distinction means teams can begin to discover better processes, techniques and tools to improve testing.

Ultimately, for me, testing is much than asserting an application meets scripted expectations, so using BDD as the sole basis for a test strategy will never deliver a robust test strategy a team will need. BDD was never designed to be a solely testing process.

Beyond BDD

As I began developing this series it just so happened that Dan North gave a talk about BDD not being testing (which I encourage you to watch: “BDD is not about testing”) in which he declares a call to arms to testers to start focusing on approaches that solve problems that BDD was never designed to solve, such as test coverage or automation.

From a sapient perspective there is a lot of great work around exploratory testing which is already being used successfully by teams in conjunction with BDD, but what about test automation. As far as I am concerned, this is a question that hasn’t been answered. Answering those questions was one of my motivations for this series. To look at the current state of popular test automation and unpick it from BDD and ask what next? If BDD isn’t the answer to automating in testing, what then takes it place?

Originally published at www.hindsightsoftware.com.