Quick guide to load testing with Siege

In a previous post I used Siege for load testing. I just wanted to give a quick introduction and explain the settings I used.

Siege is a http load testing and benchmarking utility created by Jeff Fulmer. It can be installed via Homebrew on MacOS and is available in the default repository of many Linux distributions. So it should be quick and easy to install.

Siege provides lots of control via command line options, but the following is what I used for my load testing:

siege -delay=0.5 -file=staging-urls.txt -internet -verbose -reps=200 -concurrent=15 -no-parser

Command Line Options

The only options I needed to adjust for each test was delay and concurrent.

But lets look at what each setting does and why I used it:

delay: configures a random delay between each client’s requests. The delay is in seconds between zero and the number specified. For my tests I used 0.5 seconds. This random delay makes for a more realistic test. Preventing all clients hitting the server at exactly the same moment. Siege helpfully excludes the delay from the output, so you don’t need to worry about it skewing your results.

file: allows you to specify a file containing a set of urls to use in your test.

internet: randomises the selection of urls from the file.

no-parser: by default Siege will parse returned HTML and request any css/js/images assets. This option prevents those assets being requested. Since our Production assets are served from a CDN this makes the test more realistic.

reps: the number of times the test will repeat.

concurrent: Number of clients.

verbose: outputs details of all the requests as the test runs.

File

The format for the request file is very simple. Just a plain text with a single url on each line.

https://staging-mall.finc.com/products/439
https://staging-mall.finc.com/
https://staging-mall.finc.com/t/protein
https://staging-mall.finc.com/products/ranking
https://staging-mall.finc.com/campaigns/12
https://staging-mall.finc.com/products/3

As mentioned in my previous post, I used NewRelic to see what requests contributed to most of the server load. I selected a subset of urls that represented the major load. In our case the Products#Show requests are the most common so I added two into the file to weight the requests towards it.

Output

The verbose setting means that we get a stream of output as the test runs. It shows protocol, http status code, time a request took to complete and bytes transferred:

HTTP/1.1 200 0.62 secs: 9404 bytes ==> GET /products/ranking
HTTP/1.1 200 1.80 secs: 13602 bytes ==> GET /campaigns/1
HTTP/1.1 200 0.86 secs: 16327 bytes ==> GET /products/3
HTTP/1.1 200 8.17 secs: 29438 bytes ==> GET /
HTTP/1.1 200 0.70 secs: 12646 bytes ==> GET /products/439
HTTP/1.1 200 11.07 secs: 29435 bytes ==> GET /
HTTP/1.1 200 0.40 secs: 9401 bytes ==> GET /products/ranking
HTTP/1.1 200 2.14 secs: 13608 bytes ==> GET /campaigns/12
HTTP/1.1 200 0.78 secs: 12688 bytes ==> GET /products/439
HTTP/1.1 200 0.49 secs: 9399 bytes ==> GET /products/ranking

When running tests with lots of concurrent clients you might see the request time grows as the server struggles under the load and as requests get backlogged. You might also start seeing timeouts or errors from the server.

The final result are output at the end of a test and are simple to understand:

Transactions: 4000 hits
Availability: 100.00 %
Elapsed time: 860.53 secs
Data transferred: 58.49 MB
Response time: 1.96 secs
Transaction rate: 4.65 trans/sec
Throughput: 0.07 MB/sec
Concurrency: 9.13
Successful transactions: 4000
Failed transactions: 0
Longest transaction: 18.09
Shortest transaction: 0.26

In conclusion, using Siege with these settings is a very quick way to conduct load testing and track the effects of various server configurations.