Reducing Automation Test Maintenance Cost using Test Data Parameterization
Introduction
In some cases, tests involving time constraints are cumbersome enough to automate. Let’s say you are making a marketplace application, there will be a voucher feature that users can use when making transactions, in the back office of the application, the voucher must be made to appear on the user’s application, the voucher displayed to the user is a voucher that has an active time less than or equal to the current time, so when the voucher active time is configured on 28th October, the user can see the voucher on October 28th while on October 27th the voucher would not be displayed. Based on the requirement, if we want to test the voucher feature, there are at least two test cases that can be run. The first test case is when the active time of the voucher is more than the current time, the application will not display the voucher and the second is when the active time of the voucher is less than the current time, the application will display the voucher.
To make the automation test for the two test cases, we can insert two voucher data into the database, namely voucher A and voucher B with different active times, if we create the automation test on October 28th, the active time for voucher A will be configured on November 28th, while the voucher B on September 28th, this data will be inserted into the database every time the automation test will be run and then deleted after the test is finished. In the first month, when you run the automation test repeatedly, maybe the results are in line with your expectations, but when you run the test on November 28th, the first test case will fail, why? because the active time in voucher A now less than the current time so the voucher will be displayed to the user, therefore we need to update the active time data for voucher A to be more than the current time so the test results can back in line with the expectations. Imagine if there are many test cases like this, our maintenance cost will be high since at a certain time we have to update the data before the test can be run again. To reduce the maintenance cost, time-related test data must be made dynamic by utilizing parameterization.
The Idea
In general, the parameterization of custom test frameworks is indicated by a pattern string. Some of them are indicated by the string “{{parameter}}”, “$ parameter”, “#parameter”, or many other patterns depending on the framework maker. The purpose of this pattern is to make it easier to identify so it can be replaced with a value generated by a function or value that matches the key in the properties file. In its implementation, the parameterization pattern can be easily recognized and processed further using regular expressions.
There are at least 4 important capabilities that must be addressed by time data parameterization to solve similar tests described in the introduction section. Here are the capabilities that we want to have in our framework:
- The time data should be able to set relative to the current time, for example, one day from the current time or one day before the current time
- The time data should be able to set static to a certain time, for example, it can be set at 17.00 or 03.00 by today
- The time data should be able to set based on the current time
- The time format can be changed as we want.
Implementation
Based on the previous section, I have implemented the following parameterization in my test framework, the patterns were designed to fulfill all the possible scenarios during the test.
The value in “DATE()” is used to specify the time format that we want to present, the value in “NOW()” is used to set the time static to a specified time, and the value in “OFFSET()” is used to set the relative time. The unit of time used is represented by seconds in order to address other possible tests with a smaller time constraint.
The first pattern means the time will be static at 01.00 with an offset of -86400 seconds or equal to one day before. The static time is calculated from 00.00 so if we write 3600, the time will be set one hour from 00.00.
For the second pattern since we didn’t specify the static time and only specify the relative time then it means the time will be 2 hours ahead from the current time.
Last for the third pattern means the string will be replaced with the current time since we left the static time blank and not specify the relative time.
Now, from all the patterns, let’s say we run the automation test at 18:00:00 on 28 October 2020, then the first pattern will be replaced with “27–10–2020 01:00:00”, then the second pattern will be “28–10–2020 20:00:00”, and the third pattern will be “28–10–2020 18:00:00”.
If we go back to the problem in the introduction section, we can prepare two voucher data that will be inserted into the database with the first voucher and second active time value shown below :
Here we expect that the active time of the first voucher is at 00.00 with offset one month after the current date and for the second voucher we expect that the time will be replaced at 00.00 with offset one month before the current date, so now whenever we run the test it will always inline with our expectations.
How To Implement
To implement time data parameterization in a test framework, a regular expression is needed to recognize each pattern and extract the value of each value contained in the string “()”. Below is the regular expression used for capturing the three patterns,
With this regular expression, for the first pattern, there will be 3 groups of strings captured (date format, static time, relative time), the second pattern is 2 group strings (date format, static time), and the third pattern is 1 group string (date format). Each group of these strings can be further processed to produce the appropriate output. Below is an example of a function that used to process the value of the first pattern, we only have to parse the current time to the sourceDate variable, and each value captured by the regular expression to the format, setTime, and offset variables.
Final Words
Tests that involve time can make the maintenance costs of automation tests high. To reduce the cost, one way that can be used is to implement parameterization on the time data. With capabilities such as setting the time relative to the current time, setting the time static to a certain time, setting the time based on the current time, and changing time formats make this time data parameterization very powerful to use when making automation tests. Besides that, implementing it to the test framework is also quite easy in practice so let's implement it right now.
