Testing Traffic Rules: What Would Test Analyst Do (Part I)
You can test literally anything that surrounds you. Including traffic rules.
As an example, I took one section of the Ukrainian traffic rules to show the tester’s thinking process as well as analyzing and testing any product. The steps we need to go through are below.
- Reading test basis and extracting requirements from it.
- Test analysis (deciding which technique will help to test traffic rules section to meet all the requirements).
- Test implementation.
- Test execution.
- Report.
Part I of this article will represent the first two steps. In Part II I will briefly describe 3–5 steps. Follow me on Medium and get notified once Part II will be available.
1. Test Basis and Requirements List
As a part of a test basis, we usually get a user story with its description and requirements, diagrams, documentation, or all at once.
In our case, section 4 of the Ukrainian traffic rules is “Pedestrians duties and rights”.
This step includes reading and analyzing requirements with their acceptance criteria list, and other important things that might occur. It’s a rare case when a user story has an acceptance criteria list. Most of the time it’s either plain text that needs to be transformed into a readable list or the absence of unambiguous and clear requirements of any kind. In that case, QA/TA work becomes a little bit tricky. But this is a topic for another article.
Let’s go back to traffic rules as our test basis. After we read the test basis we need to have a requirements list as a base for our expected results in our future test cases.
2. Test Analysis
My favorite part. At this step, the Test Analyst should decide what testing techniques might be applied in this particular case.
Most of the traffic rules paragraphs can be tested with use cases. But some paragraphs might require combinatorial technique. Later I will explain, why we couldn’t apply the pairwise technique to significantly reduce the number of test cases. And why the same combinatorial technique might provide us with a different amount of test cases.
In the second part of the topic, I will share my vision of how to select the right set of parameters and test design techniques.
I don’t want to deep dive into the process of finding parameters. Instead, I just list them. So, after I read the “requirements” of our traffic rules I extracted such parameters (variables):
- who;
- where;
- when;
- transport type.
Next step is to define values for each parameter:
- who [adult, kid, column of fours (adults), column of fours (kids)];
- where [carriageway, roadside, sidewalk, continuation of a sidewalk, crosswalk];
- when [day, night];
- transport type [motor vehicle, special transport, bike].
Now, it’s time to think about which test design technique will fit better in terms of time consumption and reasonable coverage.
Imagine, if we used a combinatorial technique with all four parameters and their values. In that case, you would get 120 unique cases.
When you look at this number, you realize the time consumption and reasonable coverage would be drastic. Depending on your goals, we might need such an exhaustive approach. But let’s check what we can do to decrease the number of combinations.
To do so, we need to turn on the Test Analyst mode and ask ourselves questions about the right parameters combination.
What I did here, I split these parameters into two groups.
First group (40 combinations): [who]-[where]-[when]
Second group (60 combinations): [who]-[where]-[transport type]
Total: 100 cases.
This is the result of test basis analysis and its requirements. The most creative part and a hard one at the same time. But you will get better at this over time.
Still, we had 120 combinations, now 100. The number of lines didn’t become tangibly smaller. Nevertheless, it also takes place. If it’s critical to check the combination where only one parameter’s value has been changed then go for it. But don’t forget to prioritize test cases and exclude impossible scenarios if applicable.
There is a false belief that pairwise is some kind of a silver bullet. But we cannot apply it every time. It is a Test Analyst’s job to recognize the suitability to a concrete situation.
If we would need to check pairs inside one test case, the Pairwise technique is the right choice. It provides us with 20 cases with 120 pairs (70 of them are unique). But don’t be too excited about it. And I’ll explain why.
Despite the fact we have only 20 cases, it’s doesn’t suit our goal to test all requirements. Hence, fewer test cases is not always a panacea. The generated pairs do not necessarily bring the needed effect. For instance, the pair [where]-[when], on its own is not efficient as far as we need an actor here (pedestrian and transport type) to make it work as we need.
“ Fewer test cases is not always a panacea.”
As you might already notice, test analysis requires a lot of thinking process. My goal was to show you the whole journey of how to extract parameters and their values. To decide, which techniques are applicable and which are not. And finally, to get a set of test cases with efficient coverage which will save time during test case execution. Plus, you get more understanding of the product and awareness of all possible scenarios. As you know, early QA’s involvement and impact are always cheaper.
As a Test Engineer, you are prepared and what is even more important, you can come up with this valuable information for your team members and project managers.
“Early QA’s involvement and impact are always cheaper.”
In Closing
To get those tables I used Slothman online service. Another option is the console-based tool AllPairs by James Bach. Let me know in the comments what tools you are using for this purpose.
To be precise, the tables you saw above in this article are not actual test cases. Bare with me to find out how I managed to implement test cases. In Part II you will find out how to get traffic rules reasonable test coverage.
