Web REST API Benchmark on a Real Life Application

Working as a web freelancer I am interested in how different frameworks and technologies perform, but the majority of the benchmarks found over the internet are considering just the Hello World example.

When you are building a real life application there are more aspects to take into consideration, so I decided to run a complete benchmark between the most popular frameworks and technologies.

Alongside the performance, I was interested in how easy is to achieve specific tasks in each framework and what is the cost of scaling the application performance.

Who are the candidates

  • Laravel 5, PHP 7.0, Nginx
  • Lumen 5, PHP 7.0, Nginx
  • Express JS 4, Node.js 8.1, PM2
  • Django, Python 2.7, Gunicorn
  • Spring 4, Java, Tomcat
  • .NET Core, Kestrel

What are we testing

We are interested in the number of requests per seconds each framework achieves on different server configurations alongside how clean or verbose the code looks like.

How server configurations look

We are also interested in how each framework scales its performance and what is the price of achieving such performance. This is why we’ll test them on 3 different server configurations using DigitalOcean:

  • 1 CPU, 512 MB - $5 / month
  • 4 CPU, 8 GB - $80 / month
  • 12 CPU, 32 GB - $320 / month

What are we building

We want to test a real life application example so we’ll basically build a Web REST API that exposes 4 endpoints, each one having a different complexity:

  1. Hello World - simply respond with a JSON containing Hello World string.
  2. Computation - compute the first 10.000 Fibonacci numbers.
  3. Simple Listing - we have a MySQL database containing a countries table and we’ll list all the countries.
  4. Complex Listing - we add a users table alongside a many-to-many mapping between users and countries and we want to list all users that visited France, alongside all countries each one visited.

For building the last two endpoints we’ll use the tools each framework provides to achieve our goal in the easiest possible way.

How are we testing them

For testing them we’ll use both wrk and ab HTTP benchmarking tools in order to check if we get similar results and also variate the concurrency of the requests so each technology can achieve its maximum potential.

These tools will run on their own droplet created on DigitalOcean so they don’t compete on server resources with the actual API application.

Also, the server used for placing the test requests and the one used to run the application are linked using their private IP so there won’t be any noticeable network latency.

Benchmark results

Below you can see the results grouped by each endpoint and also you can check on a single chart how each framework scaled on different server configurations.

How the API is built

Below are the actual controllers used for each framework to make an idea about how the code looks like. You can also check the whole code which is available on Github.

Laravel and Lumen with PHP

Express JS with Node.js

Django with Python

Spring with Java

.NET Core

Conclusions

Having in mind that in a real world application almost all the requests interact with the database, none of the choices are bad and all of them could handle the requirements of most web applications.

However, the performance of the Node.js with Express JS is quite remarkable. It competes with technologies such as Java and .NET Core or even outperforms them and combined with the simplicity of Javascript ES6 which you can natively use with Node.js 8, it provides so much power.

Regarding application scalability, the best performance per cost was gained on the middle size server. Adding 12 cores and 32 GB of memory didn’t help too much. Maybe, in this case, the bottle neck is somewhere else or it requires fine tunings to unlock the full server potential.

What do you think?

If you found the results interesting please click the 💚️ button so other can see them, too. You can find all the source code here:

https://github.com/mihaicracan/web-rest-api-benchmark