Performance testing of web applications
To assure that your application will not crash during the big load of users after you go live, I strongly recommend preparing and executing performance tests. In this article, I would like to dive into this topic to discuss some important aspects of performance testing, give some tips and hints, and outline the solution which we are using in Actum. There are basic questions to answer where, what, when, and how to test.
Let’s start with where to execute performance tests. If you are executing your performance tests for the first time on the production environment then it is already too late, because you are losing space to optimize the performance of your application and infrastructure before go-live. Sure, you need to run performance tests on the production environment, but just a couple of times to evaluate performance after deployment. It is also not a good idea to execute more time performance tests on production to not reduce the usability of the website for end-users of the application.
There should be a dedicated environment for testing which should have the same or at least similar configuration hardware and software like the production environment, where you will properly configure your tests and also observe the behavior of your infrastructure to see if there are any problems occurring during the execution of your performance tests. If needed your performance tests can lead to changes in the configuration of infrastructure.
We already spoke about the fact that performance tests should be executed sooner than after deployment on production. So now we are getting to the question of when to execute performance tests. I would say it is dependent on the frequency of your deployment, but if we have 2-week sprints, I would say it is good to perform it once per sprint and execute it on the UAT environment after deployment. The reason is that UAT should be the same environment in the meaning of configuration and infrastructure like production. You are not deploying there daily, and you are deploying properly tested, finished functionalities from a development perspective. So basically, on the UAT environment, you have almost a production environment, but the only difference can be in the content between environments.
How do we know that our performance of the application is fast enough or if API sends a response quick enough? That is another key question for performance tests.
If you are doing some upgrade or rework of existing application, then you can obtain data from application analytics and set criteria based on this data. If not, then you can come up with performance criteria based on some common best practices.
Another important point of view on performance tests is from an infrastructure perspective. Caching, CDNs, location of servers are other circumstances that you need to count with. They are greatly influencing the criteria of performance tests or location from where tests need to be executed.
Also, it is important to know what you are trying to achieve. If you are trying to simulate the load of users, or it is only about sending a bunch of requests to API, etc, then you may need to include some caching to your test. It is a complex topic and it depends on your case.
Another topic which I would like to touch on is reporting. You need to know what you want in terms of results, and you need to interpret results properly. Interpreting results properly is not easy especially in case of unexpected or bad results. The problem can be in test configuration, in configuration virtual machine, in the application itself, or infrastructure configuration. This analysis of cause is a painful process and it can take some time but must be done if you want to improve the stability or performance of your application.
The solution which we are using in Apollo Division unit of Actum Digital
There are many tools available for the preparation and execution of performance tests. It depends on what kind of performance tests you need to prepare. Based on this I would then choose what kind of tool to use. We are testing in Actum mainly web applications and APIs, for this reason, we chose Apache JMeter which fulfills all our criteria.
JMeter is an open-source JAVA-based application that has a well-arranged and simple GUI, where, if you know what are you doing :-), you can set up performance tests quite quickly, but don’t expect anything fancy. JMeter’s GUI is focused on a functional perspective. You can also extend JMeter functionality with plugins that you can install to JMeter to not only to have nice charts but an extension of configuration for your tests.
I will not explain here all elements in JMeter and how to set them up. For this please look to JMeter documentation where everything is described in detail (https://jmeter.apache.org/usermanual/index.html). I would like to focus more on the execution of performance tests.
The first point is that the JMeter test should be executed from the console, not from GUI. GUI’s purpose is strictly to prepare tests and maybe for attempts to execute and evaluate whether tests are doing what you are expecting. The second point is, and that is how we have it in Actum, that you should execute a test on a dedicated machine only for performance tests. I am recommending host machine in the cloud to minimize the influence of unintentional circumstances like your connection limitations, other installed applications, and so on. I also recommend the Linux operation system for this basically, because for me it worked the best.
JMeter can provide you with two important outputs. JTL file which contains basically all data from test execution like page load time, connection time, etc. This file you can then use as input file in JMeter listeners like charts, aggregate report, etc. You have also the option to generate a comprehensive HTML report which nicely presents results of test execution.
Last, but not least, is that you can integrate creation VM with the virtual machine for performance test to Azure pipeline with your test.jmx file. Execute, obtain results, and copy, for example, to Azure blob storage and then destroy the machine. This way is the most efficient from a cost perspective and is not hard to maintain.
Example JMeter test
To showcase JMeter capabilities I prepared a basic test for one page. In the test, I am simulating the load of 300 users for 10 minutes on-page. To determine if the infrastructure can handle this amount of users. This test is just a showcase so take it like this.
I am using plugin Ultimate ThreadGroup which has more configuration options and is more flexible than the standard Thread group. You can set ram up time, duration, shutdown time, or delay. You can add more threads in time or decrease the number of threads and also show the timeline of tests with active threads.
You can see the test contains one HTTP request to the page which contains configuration for URL and in advance tab also option to include embedded resources which are important to get all relevant resources for the page. Another important feature is the option to exclude some resources, for example, tracking requests like Google Analytics because it influences performance, but in the end, you cannot influence the performance of 3rd party services so why measure it.
Header, cookie, and cache manager are in test basically to simulate the user itself or in better words the browser. These items are there for caching mainly and additional configuration if needed. It is important because if these items will not be there you are sending just a bunch of requests to the server and the test would not fulfill the goal and results would be useless.
The last part is a gaussian timer which is creating a delay between requests. If there wouldn’t be this item in the test, then JMeter would send many more requests basically, then the limit would be the performance of machine and response time from the server. The gaussian timer has also an option to put there a deviation time which is creating a random part of the time in delay. You are not sending requests in waves thanks to this deviation. It is another factor that helps improve the simulation of user “behavior”.
So that was a simple test to show some basic capabilities of JMeter now let’s see the results. Below you see part of the HTML report generated from test execution to be a more accurate chart of Transaction per second for all requests which were made during testing the page. In the menu, you can see there is much more in the report. I will not comment on results, because it doesn’t make sense, because this was just an example.
