Benchmarking a Rails API

Yesterday, I ran into a wall. The benchmarking wall.

Some of you are thinking, what’s that all about ? What “benchmarking wall” ? *Sigh* Well… Have you ever tried to benchmark your Rails API in less than 30 minutes ? Open your most recent project and do it, then tell me, how much time did it take ?

Either you’ll have to write some “PerformanceTests” for your API, make them get along with you RSpec tests, configure them etc.. or run tools that do magical things, buying cloud based solutions and so on. But what if I just want to do a SIMPLE benchmark of my app, no sorcery tricks nor complicated stuff that does *extreme load balanced cloud AWS mega stress tests or bombing my App as if it had oil (petrol) ? Nothing ?

My goal here is just to see the expected response time with N concurrent requests in production, and maybe one more thing, a formatting tool to make it easy to use and extend, markdown format for Github or JSON for CI integration. Then I could track down the slowdowns linked to new features and refactor. It can also help me to confront the upcoming production environment and brace myself.

*Means nothing

ApacheBench to the rescue

This tool is simple, yet it does what it adverts.

ab -n 100 -c 10 “http://myawesomeurl.org”

Understand: test myawesomeurl.org with a hundred requests processing up to 10 requests concurrently.

Yes ! This is exactly what we talked about. Except for the formatting, although it can format in TSV (Gnuplot) and CSV with only percentages served (e.g: 50% < 42ms, 99% < 85ms). That’s a thing, yet it’s not enough. We need to go deeper.

RailsApiBenchmark

How about leveraging the power of ApacheBench ?

RailsApiBenchmark is a fresh-made gem of my own which leverages the power of ApacheBench by integrating it in a Rails API without too much effort. It adds a formatting concept with regexps to extract data from ApacheBench output, no rocket science but efficient. It is configurable to change the benchmarking tool and the server used for the benchmarking process.

Here is some screenshots of the Markdown formatted results with Gnuplot integration (basic configuration of the gem does this):

This the index, README.md in the benchmark/ folder of your project once you’ve run rails api:benchmark

Here is the README, the page you’ll see in benchmark/ folder. It lists all the routes configured to be benchmarked.


The results are listed in a table to allow fast-spotting of slowdowns.

Results for a single route, with gnuplot

And finally, the result page for a single route. You can read the average response time / requests per second and see a gnuplot graph for the concerned endpoint.

Here at Getpro, we’ll use this gem as long as we need to stick with low latency apps. Every single feature that needs to be put in production will be scanned using this benchmark tool !

While we do PR reviews on Github for every single feature, the PRs will be followed by the benchmark summary diff (see the Summary screenshot).

What do you think about this kind of benchmarking and what tools do you use to achieve a similar result ?

Check out my previous article: Why I chose Rails::Api over Grape