Achieve success with your testing by creating Test Scenarios and Test Cases accurately

When it comes to software testing, you’ve probably heard expressions like Test Scenarios, acceptance criteria, Gherkin, and more. As the testing market expands, new terminologies emerge. It’s crucial to comprehend these terms and their meanings to apply them accurately in our daily activities. The purpose of this article is to elucidate the distinctions between Plan, Scenario, Condition and Test Case. You might have used these terms interchangeably, but are they truly synonymous? Ensuring the quality of a test demands it to be an accurate representation of the use cases of the tested subject, thereby ensuring robust functionality. To achieve effective tests, meticulous planning is indispensable. For effective planning, we need to know and have knowledge about the basic concepts.

Let’s begin with the Test Plan. During the execution of a project, whether software, hardware, study or any other type of project, multiple documents are generated to plan the project. Among these is the Test Plan. It outlines the testing strategy, what will be tested, who will perform the tests, how the tests will be conducted, and the tools used. In agile software development contexts, the complete Test Plan is not infrequently used due to its extensive and bureaucratic nature. However, a simplified version can be helpful, being a document in which we consolidate all project Test Scenarios. The aim is to map the testing process that assures value and delivery quality to the client.

A Test Scenario, on the other hand, embodies a notion of what should be tested. Notably, the term “Test Scenario” was recently created by the industry and is synonymous with a Test Condition. Test scenarios describe situations to be tested for a well-defined project, identifying whether outcomes meet expectations. How are these situations described? Imagine testing a banking system. An example of a Test Scenario could be: “Test that after making a deposit, the money becomes available within the stipulated time.” Or: “Test that the card linked to the account can withdraw money.” Notably, a scenario contains a brief description detailing the functionality to be tested and the expected outcome.

Now, let’s define the Test Case. It is a set of input values, pre-execution conditions, anticipated results, and post-execution conditions developed for a specific test condition or scenario. This implies that the test execution is defined by the Test Case. A scenario or test condition can have a series of Test Cases. In the example of the Test Scenario “Test that after making a deposit, the money becomes available within the stipulated time,” the following Test Cases might exist:

Test Case 1:

• Pre-condition: Deposit into the account should be functional
• Inputs: Deposit of $100
• Expected result: The account balance should increase by $100 after 24 hours
• Post-condition: The $100 remains in the account the following day

Test Case 2:

• Pre-condition: Deposit into the account should be functional
• Inputs: A $100 deposit was made 4 hours ago
• Expected result: The $100 value should be displayed in the account as a future transaction
• Post-condition: -

These concepts can be represented in the following diagram:

With a comprehensive grasp of test strategy components, we can introduce the concept of Acceptance Criteria. These criteria are specified within a User Story, describing the expected behavior of the requested functionality, and defining the boundaries of that story’s scope.

What is the relationship between acceptance criteria and Test Scenarios and Test Cases? Acceptance criteria should serve as inspiration for defining the scenarios to be validated, however, they don’t represent all possible scenarios. Testing techniques are used to define which are the necessary testing scenarios and what is expected from them. These techniques can be boundary cases, performance cases and unexpected behavior cases.

Gherkin

Gherkin and Cucumber are well-known terms in the Software Quality market. Cucumber is a tool that reads executable instructions written in plain text format and validates whether the software function according to the specifications. Gherkin, on the other hand, consists of grammatical rules that structure text in a way that Cucumber can comprehend. Consequently, Gherkin’s format significantly facilitates the creation of automated tests. Does a Gherkin test represent a scenario or a Test Case? It does not. The purpose of a Gherkin-based test is to replicate and assess a business rule, transforming it into an automated test using Cucumber.

Despite its resemblance to a Test Case, due to its pre-condition (Given), input (When), and expected outcome (Then), Gherkin serves a different purpose. Can Test Cases be used to generate some tests? If it makes sense for your application’s validation, it’s possible. However, not all Test Cases must become automated tests, nor do Gherkin-based tests require equivalent Test Cases.

In Gherkin, the term “Scenario” represents what will be described in the subsequent lines. This inadvertently created a link with Test Scenarios and later with Test Cases. However, referring to the Cucumber documentation, we find that the term “Scenario” is used synonymously with the term “Example” (as seen in the Rule section), rather than representing a scenario or Test Case.

Understanding these concepts equips us to apply them to our reality and optimize our work, ensuring efficiency in asserting that delivery is of high quality. After all, the testing process also needs structure; and having a plan with corresponding scenarios and Test Cases provides assurance that the validation of that delivery was structured and planned using quality techniques and their applications. While it is possible to test an application without scenarios and Test Cases, we won’t have the guarantee that all validations were contemplated and executed, and there won’t be documentation as evidence. Consequently, failures are more likely to occur. Typically, those responsible to guarantee the quality of a delivery have diverse experiences and knowledge of processes and techniques. Creating a comprehensive Test Plan collaboratively helps to level the playing field, ensuring that all teams achieve similar quality in their deliveries.

As mentioned at the beginning of the article, the Software Quality market is evolving. It’s essential to ensure that these concepts remain accurate as the field progresses. After having delved into these concepts together, I hope the relationships between these terms, which are so important in software testing discussions, have become clearer.