Performance Testing: Do You Test Regularly?
Many companies do not think about the application performance until real volume comes. Obviously, it is too late to test because everybody is busy fixing the performance issues and tuning hardware.
Often, performance testing is not conducted due to the lack of qualified resources available, lack of experience or due to thinking, “It is not a time yet; we are too small.”
Performance defects are always unexpected and damaging. If you are not ready to invest into complex performance testing but you feel that it is a time to start, then this article is for you.
In this post, we will share our experience and show how to do a baseline performance testing on your project to identify issues during the development and to avoid failures that may lead to loss of money and reputation damages.
What is baseline performance testing?
In simple words, this is the testing that measures the performance characteristics of your application against some defined values or previous testing results. Often, this testing is also called benchmarking testing.
Where does the baseline come from?
Usually, it is the current state of the product/service or some figures defined by business.
What to do with the baseline?
Having the application performance baseline, and after running a performance test, we’ll be able to answer the most common and important questions from a product owner: “Is the application as fast as it was before release?” or “Will the application handle the incoming traffic if we run a new marketing company?”
Today, we will share with you how to create a simple test script using the popular and open source load testing tool Jmeter, run it from Jenkins, see the results and compare them with the previous run.
Let’s put ourselves into the product owner’s position and define the requirements or use case.
- Our scope is the home page of “test.com” website
- We want it to handle 100 users coming every minute during 2 peak hours
- We want the response time for the home page to be around 2–3 seconds
- We want this requirement to be tested from release to release
Having this information, it’s time to build the script. We will need Jmeter with “bzm — Arrivals Thread Group” thread group plugin.
There is an assumption that you are familiar with Jmeter and Jenkins tools.
- Set up target application. In our case, it is dummy sampler that represents a home page of our test.com website. We configured it to have a response time from 2 to 3 seconds.
2. Generate the load you need. In order to fulfill the requirement, we use “Arrivals Thread Group.” It allows to set the constant volume that will be held independently on the response time the target application produces. In other words, we are sure that 100 virtual users will “come” to the home page even if response time drops to 10 seconds or more. In our example, we will run the test for one minute. This value can be 2 hours according to the use case.
3. Save this script as performance.jmx
4. Run it in non-GUI mode using the command
jmeter -n -t performance.jmx -l testresults.jtl
5. Integrate your test with Jenkins CI
- Install “Publish performance test result report” plugin
- Configure the job to run the command from item 4
6. Run your test from Jenkins
We have run the same test several times; let’s take a look at the performance trend.
As you can see, the last four builds demonstrate pretty much the same response time without errors.
Now, let’s simulate degradation of application response time from 2–3 seconds to 5–6, run the test and take a look at the performance trend again.
It is very clear that the last build has an anomaly and must attract attention to anybody who looks into the report. Press an alarm button!
We have demonstrated that performance testing can be achievable, it is relatively easy to set up and it is transparent for everybody involved in the development process.
Demonstrate the trend of application performance each time you are asked about its state or include it into testing reports.
Of course, we have shared a very basic example, but principals stay the same:
- Test the most valuable and business-critical functionality
- Test often
- Automate as much as you can; you will save time in the future
For more complicated cases, you can always reach out to the Rness team or leave your comment under this post.