The I think you’ll agree with me when I say:
It’s REALLY hard to maintain load test scenario!
Well by changing your strategy, you can get out of this maintenance hell.
Change your strategy by being more agile without a single effort.
Most of the companies be like:
“The release is on the 5th lets plan the performance testing on the 2nd”
Actually it took me years to start to understand that something is wrong there. Until I realized that by following this pattern, we were able to test only a few happy path. The truth is that even if you realized that your product can’t handle the expected load you will just switch your mindset and lower your expectation.
Obviously the challenge is coming from the business target, so I can’t blame this practice.
The goal is to test performance early and often in the development effort and to test functionality and performance in the same sprint — Rajni Singh
Definition of Done is the KEY
First of all, you need to have a clear definition of done I will not explain this part, but if your team doesn’t follow this properly you have a few homework to do.
As mention by Rajni Singh in this excellent article : Real Agile Approach To Performance Testing, “The goal is to test performance early and often in the development effort and to test functionality and performance in the same sprint.”
The only way to meet the performance expectation is to:
- Define the performance objective for each API at the beginning of the sprint.
- Add the performance testing into the definition of done
A Feature can’t be accepted if it doesn’t meet the performance expectation.
But the reality is different even if you have this strong rule. This will definitely decrease the velocity of your team and unlikely bend the rule… This key to stick with this rule is to have an easy and efficient Load Test strategy, that will not add more workload to the team.
Save weeks of load test maintenance with one simple line
For this implementation, we will rely on the simplicity of RestQA. As you know RestQA is a free open source tool that allows you to write End to End test automation for your APIs. In case you don’t the setup go do it and it only requires 1 minute to install.
Our Test Automation is based on Gherkin language, a language allowing us to describe the behavior of our API using natural words.
As you can see the current scenario is testing the behavior of one or our API. What if we just want to reuse this exact scenario for our load testing.
It will require only 2 steps to do it:
- You just need to add the tag performance to your scenario as below (line 1):
2. Specify into your configuration file what is the tool that you are using for the performance testing. In the example below we are using artillery:
description: Configuration generated by restqa init
- name: local
- name: '@restqa/restqapi'
- type: html
And that’s It, once you will run the command :
You will discover that all the scenario that you tagged as performance will be translated as load test scenario compatible to artillery into the folder:
Then from there you will just need to run your load test using your artillery and custom configuration such as:
artillery -c custom-config.yml /tests/performance
I let you imagine how simply it will be to integrate this performance testing step into your continuous integration flow.
Select your load test in one command
Once your setup is ready you can select the scenario where you want to focus on all the scenario that are tagged as performance for a specific environment.
This is were you can just use the command:
restqa run -t performance
The command above will focus on running only the test that are tagged as performance and ignore all the others.
Load Test Tool integration
As of now, RestQa is generating performance test scenario exclusively from Artillery. You can use this feature side by side with the other integration tools such as K6, Jmeter, Gatling, Vegeta and etc.
Feel free to submit an issue on RestQA repository and we will prioritize this integration according to the number of request we receive. 😇
If you have been frustrated by the weakness of your load test strategy, I believe this few tips will help a lot!
Traditional strategy does work. But it works at a costly price for your team and the product quality.
Performance testing doesn’t have to be a headache. There is a lot of tool out there but it’s all about how you use them into your flow.
Like I say all the time: Software Quality is a long term strategy!
This small tips will help you to get started at a lower cost and you’ll see a lot of benefit on your team. RestQA is built to help you on this journey.
Feel free to take a look at RestQA. Let us know in the comment how you feel about this feature.