Load testing with JMeter.
Do you know how many users can safely browse through your website without taking it down? To me, it is one of the most important metrics you need to know about your website. But the truth is, more often than not (because of reasons such as “the traffic is usually low”, “we have X servers just in case” (air-quoting here), or simply due to lack of time), we tend not to pay attention to such metrics.
In 2019, a celebrity tweeted a link to one of the platforms Vizzuality develops. Although this was amazing publicity for us, our client, and their website, it also required some fast action from our dev team to keep the website live as thousands of people clicked on the link this celebrity had shared.
A few months ago, we received news that a press release was being written about one of the features on this same website the celebrity had tweeted about. There was a chance of another spike in user traffic — and this time, we wanted to be prepared for it. So, we decided to load test the website. This way, we would know what our limits are, and we could act on this information if we saw the increase in traffic approaching the limits.
What is load testing?
Load testing is a subset of software testing focused on measuring a system’s response to high-demand input. Load testing can be put into practice in multiple ways, taking into account the system being tested. For example, we could load test:
- a website to determine how it behaves when facing a spike in traffic;
- an API to determine performance bottlenecks;
- a GUI to see how it handles multiple users interacting with it at the same time or how it handles very complex objects (e.g., one could test Google Docs, with the intent of understanding how many people can use the same document at the same time).
There are many tools you can use for load testing. And, as in many other use cases, you can find tools ranging from open-source, free and simple to use, such as Siege and JMeter, to feature-packed, premium tools such as SmartBear’s LoadNinja. For this example, we will use JMeter since it was free, and its features match our use case.
Load testing with JMeter.
JMeter is an open-source tool built with Java that provides a GUI for creating load testing scripts. JMeter might look scary at first since its UI is quite complex, but you’ll be able to find tutorials online to guide you through the process. This one, in particular, is a good place to start.
For our use case, we decided to keep the test case simple. For that, we considered users would access the main page of the website one single time. Then, we consistently incremented the number of concurrent users (Number of Threads, in JMeter lingo) bombarding the website with requests. When the website started having trouble handling all requests, we ended the test.
We were able to understand that our infrastructure was able to handle 8,000 concurrent users accessing the main page in 1 minute without significant issues. With 10,000 concurrent users, we saw a small increase in response times. And with 12,000 concurrent users, we experienced slowness and even some periods of unavailability.
Things to keep in mind.
Load testing provides useful information for better decision-making. But there are some caveats that you should consider.
Be extra careful when load testing live environments.
If you are load testing a live website or web server, you should be extra careful when testing the limits of your website. Keep in mind that any downtime caused by the tests will also affect users using your platform.
Define a flow that mimics what your users usually do on your website.
While load testing, the more your test case reflects actual user behaviour, the more accurate the test results will be. For our use case, a single-page request is a good enough reflection of our users’ behaviour. Most of the time, our users access the platform one time and then spend some time exploring the data they are viewing.
In most cases, though, it’s probably more realistic to have a more complex flow. For such cases, you might want to set up a test case that properly simulates a realistic flow of user interactions on your website. If, for instance, most of the traffic comes from API requests, you can take a HAR snapshot of requests from a user session (navigating and clicking around in the browser as you assume a user would do) and then feed this to a test script that would replay the API requests with the original time gaps.
The main takeaway here is: the more you invest in making your test script realistic, the more accurate the load testing results will be.
Consider the limitations of your own hardware.
While load testing, you are bound to hit some limitation: in many cases, these will be the limits of the server you are load testing, but you might actually reach the limits of your own hardware (as in, your computer can’t make that many requests at the same time).
To avoid such pitfalls, keep an eye on the performance metrics of the servers being load tested (CPU usage, memory usage, etc). You should observe metrics fluctuating, reflecting the execution of the load test. This way, you can tell whether your load testing is hitting the limits of the server or your hardware.
Conclusions.
Load testing allows us to grab valuable information about the limits of the websites we build and share them with our clients. If a major public event is planned, we can prepare for it and give our clients confidence that their website will stay live and online. Knowing our limits also enables us to make more informed infrastructure decisions in the future.
Fortunately, this year, we haven’t experienced any intense spikes in the number of simultaneous users. But, if another celebrity chooses to share a link to one of our websites, we are prepared and ready for any spike in visitors.
Henrique is a backend software engineer at Vizzuality who implements systems that collect, process and analyse data from different sources. Henrique loves hiking, camping and walking on the beach.
