Common Pitfalls When Writing BDD Specifications
I’m a Quality Evangelist for a billion dollar company. Here are the common mistakes I see when teams try to adopt BDD.
BDD (Behavior-Driven Development) is not a new development methodology, but it is still misunderstood by many in the industry. I hope to clarify some common misconceptions and pitfalls that befall many a new practitioner. If you are not familiar with the concepts behind it, I invite you to read the seminal work of Dan North, the creator of BDD, on the subject.
Embedding Technical Details
One of the common issues I see is the embedding of technical details inside of steps. Typically an overzealous developer, determined to reduce “duplication” or promote “step reuse” will write or rewrite steps to include direct XPaths, CSS selectors, or other “technical” details in the specifications. Such as:
This practice, left unchecked, can effectively re-create entire functional testing frameworks as developers looking to “improve” the system add more and more features. Noticing that you are copy/pasting the same XPaths all over our feature files, a central repository of XPaths and CSS selectors is added. This process is repeated ad-naseum. You don’t have the resources to maintain your own custom functional testing tool, so don’t try.
The worst part of this approach is not the time lost spent building the framework. The problem is that this approach misses the entire point of BDD. BDD is a way of driving collaboration and communication between business and developers. It is not a testing process, it is a system for driving the design of the system based off of business behaviors. Your business partners do not, and should not, understand CSS selectors and XPaths, or whatever implementation details you decide to shoehorn into your specifications. They care about the value that these things are going to bring them. Put that value in your specifications.
Rewriting the example above to focus on the business value we want to deliver, we get:
Ahh, so much nicer. Not only is our specification much shorter, it is clear what is going on at a glance. Our business users don’t care about how we find and interact with the elements on the page, they only care that users that exist within the system can log in, and users without accounts cannot.
While you may think that the first example I showed you was not that bad, I have seen the pitfalls of going down this road, and it is not pretty. Feature files hundreds or even thousands of lines long, with extensive set up and tear down steps described in explicit detail. Business users eyes glazed over when the stories were reviewed- eventually the reviews stopped happening all together. If your customers can’t read the specification that you wrote (with perhaps minor clarifying questions) then you aren’t doing your job. Sit down and rewrite the examples. Better yet, write the story with your business users in the room, making sure to use language that you all understand, and then you won’t have to worry about creating 1000 line plus nightmare features.
Describing the Implementation
While not as bad as embedding explicit technical details in your features, describing the implementation can be harmful to your system as well. Your stories should focus on the “What”, not the “How”.
For example, a user does not “click the submit button”, they “submit their resume”. This is important not only because it better describes the problem at hand, it allows the technical specialists the freedom to determine the best implementation of the feature.
As I write this article, the “Save” button is conspicuously absent, rather the article auto-saves as I type, alerting me obviously, but non-obtrusively in the upper left-hand corner as soon as my changes have been persisted. The experience to me, the end-user, is wonderful, as I am not doomed to compulsively click the save button every few seconds (as I often do), but rather am allowed to fully immerse myself in the freedom of writing without worrying about losing my work.
By specifying the implementation up front, we don’t allow the problem to be considered and worked through as it is solved, limiting our application before we have even written a single line of code.
As an example, take a look at this specification:
On the surface is can appear well written. But many of the steps make specific reference to implementation details. It is clear that this scenario is written from the perspective of a web page, but what if that is not the best implementation? What if our users have access to virtual assistants that can take instruction by voice commands? Even if a web page is the best possible implementation, we are now binding our tests to interact with that web page. What if most of the web functionality is already tested, and this test is best written against an API? Or against the code directly as a unit test? Because we bound ourselves to a specific implementation, instead of describing the intent, we are not free to implement the page, or the tests, in the best possible way.
Putting on our BDD caps, we sit down and dutifully modify the scenario and get:
Much better. We are still describing what is needed, but not how we will get there. Our scenario is more concise, yet still describes the necessary facts. We now have the space we need to maneuver, and can implement the feature in the best possible way.
Training ourselves to write in this fashion can be difficult. I defer to the eminent Dan North in this regard. His excellent article on domains describes the process:
For any statement, you can chunk up by asking “Why..?” or “What for..?” questions and chunk down by asking “How..?” questions. The further you chunk up, the broader your perspective becomes, and the further you chunk down, the more detailed.
If you find yourself writing technical details or describing the implementation of the system, stop, and ask yourself “Why?”.
An important aspect of BDD that is often overlooked by ameteur practitioners is that is it a development process, not a testing process. Organizations often declare that “we are doing BDD now” and hand over the tools to their QA department. Buy-in from development and business partners is not sought, and BDD just becomes another buzzword filling the corporate malaise.
Don’t be this way.
If you are in QA, rope in your developers and business partners; bribe, threaten, or lure them in by any means necessary. If you are a developer, ask your QA what scenarios or edge-cases the specification is missing. If you are a business partner, show your Developers that you trust them to implement the right features and give them the autonomy to do so; don’t attempt to micromanage.
BDD is not an excuse to throw anything over the wall to another department, whether that department is Development, QA, or the business. Don’t leave your business partners out to dry and demand they start writing in Gherkin on day 1. BDD is a collaborative effort that all areas have a vested interest in. I’ve seen the problems that arise when BDD frameworks are used by teams that have no idea how to actually do BDD, and it’s not pretty.
You will not reduce rework and improve your throughput without getting everyone on the same page. Collaboration is the first step, BDD is the second.