Performing Realistic Load Tests
When trying to simulate a realistic load test, the machine you used to develop your test script with is not enough. In that case, you need to set up a test rig.
Introduction
Load Testing is a bit of an art. Having your test scenarios written is only half of the story.
A Test Rig is necessary to be able to simulate real users as closely as possible. Indeed, you need:
- To be able to scale up to support many users, sometimes tens of thousands of them;
- To be able to simulate users from various geographic locations;
- To be able to simulate the network those users are connecting from.
On top of that, when load testing your application, you want to understand what is happening under the hood. This is where monitoring is handy.
Generating a Large Load
Decent performance testing tool should be able to scale up the number of users by spreading the load across multiple servers. A typical set up is shown below:
The testing tool connects to a Load Controller. The Load Controller is then connected to 1 or multiple Load Generator. The Load Generators are hosted on separate servers and are in charge of executing the scenarios. If you are simulating 3,000 users and have 3 load generators, then each load generator will simulate 1,000 users.
Some tools such as JMeter do not separate the Load Generator from the UI:
Here, the terminology is slightly different:
- JMeter Master: the machine running the JMeter GUI;
- JMeter Slave: running jmeter-server which is executing the scenarios.
Geographic Location
If you have a global e-commerce web application, your customers are coming from various regions around the globe. Because of the difference of the distance, your user experience will be different.
The following table shows the latency between 2 endpoints:
As you can see, this is not negligible. That’s why new protocols such as SPDY from Google or HTTP/2 have been developed to partially solve those latency issues.
To simulate users from various locations, you can simply install your Load Generators in those various regions. The easiest way is to actually deploy cloud servers in those regions.
Network Virtualization
Accessing a web page from a mobile phone or from a desktop machine at the office does not give the same performance. On one end, you may be using a slow connection (GPRS, 3G) and on the other end, you may be using a leased line with very slow latency.
Depending on the network technology used, your user will have a different experience. For instance, look at the speed and latency difference for some common networks:
There are many other elements that affect network transfer speed such as packet loss.
To simulate networks, you can either use a network appliance such as iTrinigy Network Emulator or software based. Obviously, the appliance will have a more realistic reproduction of real-life network behaviour.
Monitoring
Lastly, you may want to collect information from your server(s) such as database performance (response time, processing time) or server load of each server behind a load balancer to see if the load is equally distributed. To do that, you need to collect metrics using a monitoring tool (such as performance counters on Windows platform). Once done, you can correlate your load test performance to those counters and potentially uncover the source of the performance bottleneck. The load controller is generally in charge of collecting this data:
Conclusion
The setup of a test rig is not an easy task but once done, little change is required. However, it is quite important to know what your audience is (where they come from, what devices they use) as the load test needs this to generate a realistic load.
