Timing HTTP Requests with cURL

Sometimes you just need to quickly benchmark how fast a page can be loaded (or fetched to be more precise). For these cases, cURL is a great option for timing HTTP requests.

$ curl -L -s -w "%{time_total}\n" -o /dev/null http://www.google.com/
0.095

Want a few more datapoints?? Thanks to ZSH, it’s easy to just loop around it:

$ for i in {1..3}; curl -L -s -w "%{time_total}\n" -o /dev/null http://www.google.com/
0.507
0.077
0.077

And if you’re a bash lover:

$ for i in {1..3}; do curl -L -s -w "%{time_total}\n" -o /dev/null http://www.google.com/; done
1.079
0.124
0.106

Default behavior on cURL is GET, but you can do POST, DELETE, PUT and more complex requests. If you’re not familiar with cURL, best place to start is the manpage.

Besides “time_total”, curl also provides other timing, like “time_namelookup”, “time_connect”, etc. Checking a post by Joseph, I remembered that curl supports formatted output. This way we can create a “template” for our HTTP timing test:

\n
time_namelookup: %{time_namelookup}\n 
time_connect: %{time_connect}\n 
time_appconnect: %{time_appconnect}\n 
time_pretransfer: %{time_pretransfer}\n 
time_redirect: %{time_redirect}\n 
time_starttransfer: %{time_starttransfer}\n 
— — — — — \n 
time_total: %{time_total}\n
\n

Assuming the format file is named “curl-format”, we can execute a request:

$ curl -L -w "@curl-format" -o /dev/null -s http://www.google.com/
time_namelookup: 0.416
time_connect: 0.435
time_appconnect: 0.000
time_pretransfer: 0.435
time_redirect: 0.000
time_starttransfer: 0.488
---------- 
time_total: 0.491

Where:

• -L follow location header, in case page has moved
• -w “@curl-format” tells cURL to use our format file
• -o /dev/null redirects the output of the request to /dev/null
• -s tells cURL not to show a progress bar
• http://www.google.com/ is the URL we are requesting

The timings are DNS lookup, TCP connect, pre-transfer negotiations, start to end of transfer, redirects (in this case there were none), time to first byte, and total time (last byte), respectively.

Looking for something a bit more “complete”? You can always try Apache Benchmark:

$ ab -n 3 http://www.google.com/
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking www.google.com (be patient).....done
Server Software: gws
Server Hostname: www.google.com
Server Port: 80
Document Path: /
Document Length: 10928 bytes
Concurrency Level: 1
Time taken for tests: 0.231 seconds
Complete requests: 3
Failed requests: 2
(Connect: 0, Receive: 0, Length: 2, Exceptions: 0)
Write errors: 0
Total transferred: 35279 bytes
HTML transferred: 32984 bytes
Requests per second: 12.99 [#/sec] (mean)
Time per request: 76.999 [ms] (mean)
Time per request: 76.999 [ms] (mean, across all concurrent requests)
Transfer rate: 149.15 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 19 21 1.8 22 22
Processing: 50 56 5.3 59 61
Waiting: 46 51 4.0 53 53
Total: 73 77 5.0 79 82
Percentage of the requests served within a certain time (ms)
50% 76
66% 76
75% 82
80% 82
90% 82
95% 82
98% 82
99% 82
100% 82 (longest request)

Originally published at overloaded.io.