Unveiling the Misunderstood Practice of Executable Documentation in Quality Engineering: Debunking Gherkin Misuse and the Pitfalls of Swagger Design

As we continue in our series of Quality Engineering tooling, we continue from our previous article Observability the last choice. In my latest poll, I was surprised to… | by Alejandro Sanchez-Giraldo | May, 2023 | Medium.

In the realm of quality engineering, the practice of executable documentation has often been misunderstood and misused. Two prominent examples of this are the incorrect utilisation of Gherkin, a widely adopted business-readable domain-specific language, and the inadequate attention to quality when designing Swagger documents, a popular tool for documenting APIs. This article aims to shed light on these misconceptions and highlight the importance of proper implementation to achieve effective executable documentation.

robot reading a book, digital art by DALL-E

Understanding Executable Documentation:

Executable documentation is an approach that enables stakeholders to access, understand, and validate system through human-readable, executable specifications. It fosters collaboration between teams, resulting in enhanced clarity, efficiency, and quality of a product.

The Role of Gherkin in Executable Documentation:

Gherkin is a widely used language for defining behavior in a structured and readable format. However, its misuse can lead to confusion and misinterpretation. Here are some common pitfalls to avoid when working with Gherkin:

How not to use Gherkin by 3QLabs

• Overuse of Scenario Outlines: Scenario Outlines in Gherkin are meant to provide a template for repeating scenarios with different inputs. However, excessive use of Scenario Outlines can make the documentation convoluted and difficult to maintain. It is essential to strike a balance between reusability and clarity.
• Lack of Contextualisation: Gherkin scenarios should be written in a manner that clearly conveys the intended behavior. Providing sufficient contextual information, such as preconditions and expected outcomes, is crucial for comprehensive understanding and accurate implementation.
• Neglecting Step Definitions: Step definitions bind Gherkin steps to automation code, allowing the executable documentation to be executed as automated tests. Inadequate or incomplete step definitions can render the executable documentation useless. Maintaining a robust library of step definitions is vital for the effective implementation of executable documentation.
• Transactional Definitions: Gherkin scenarios are intended to describe a user behaviour and provide examples to ensure technical and non technical team members, to help them understand what needs to be built. Many people use it to describe a step by step and actions of a user, creating long scenarios that are brittle and hard to read.

The Quality Challenge in Swagger Design:

Swagger is a widely adopted tool for designing, building, and documenting APIs. However, the quality aspect of Swagger documents is often overlooked, leading to several issues:

• Incomplete and Inaccurate Information: Swagger documents should accurately reflect the API’s functionality, data structures, and response codes. Incomplete or inaccurate information can mislead developers and hinder effective integration with the API.
• Lack of Consistency: Consistency in naming conventions, data types, and error handling is crucial for a well-designed Swagger document. Inconsistent naming or data types can introduce confusion and errors during API implementation.
• Insufficient Validation: Swagger documents provide the opportunity to define request and response schemas, but their validation is often overlooked. Lack of validation can result in unexpected behavior and potential security vulnerabilities in the API.
• Reliance on auto-gen tools: While auto-generated solutions for generating Swagger docs exist in various programming languages, it is important to avoid complacency and solely relying on these tools without careful review and enhancement. One of the primary advantages of executable documentation lies in its ability to provide concrete examples for different scenarios. By investing time and effort into reviewing and uplifting the generated documentation, we can ensure its accuracy, completeness, and alignment with the desired outcomes.

Best Practices for Effective Executable Documentation:

To harness the true potential of executable documentation and avoid the pitfalls mentioned above, consider the following best practices:

DTO to Swagger

• Collaborative Approach: Involve business analysts, developers, and quality engineers in the creation and review of executable documentation. This fosters shared understanding and ensures accuracy and completeness.
• Clear and Concise Language: Use Gherkin and Swagger documents to provide clear, concise, and unambiguous information. Avoid technical jargon or excessive verbosity that can hinder comprehension.
• Continuous Maintenance: Keep the executable documentation up to date, reflecting changes in requirements, business logic, and API specifications. Regularly review and update step definitions, Gherkin scenarios, and Swagger documents to maintain their relevance and accuracy.

Conclusion:

Executable documentation, when properly implemented, can greatly enhance collaboration, understanding, and quality in software development.

References:

Simplifying Cross-Platform Cucumber Scenarios | 3Qi Labs

spring boot — How to annotate DTO so that it shows up in SwaggerUI Schema? — Stack Overflow