9 PHP runtimes: performance benchmark

There are some fairly new players in the PHP runtimes team. It’s interesting to understand how good they are in comparison to well-known and widely used.

The main questions I wanted to get answers are:

• How many requests per second the runtime can process?
• What’s the average response time each runtime can provide?

To have fair competition, all runtimes are given the same resources. Almost. If the runtime consists of only 1 container(ex. Nginx Unit), it is run with 1CPU and 1GB of RAM. If the runtime consists of 2 containers(ex. Nginx + PHP-FPM), each container is run with 1 CPU and 1GB of RAM.

The testing environment

• Each runtime is run in 1 or 2 Docker containers.
• Each container has 1 CPU and 1GB of RAM.
• The load testing tool is in the same Docker network

Testing application.

Let’s try something more complex than single-file Hello-World. So, we have:

• Symfony 7
• 1 controller
• 1 view
• no database or any other external service

Symfony provides the runtime component. According to the official doc it “decouples the bootstrapping logic from any global state to make sure the application can run with runtimes like”. It means, that you can develop the application using any runtime you like, but in production run the most performant. So, we can run the application not only with traditional servers, but with libraries like Swoole, AMPHP, and ReactPHP. Looking ahead, the last two were excluded from the competition.

Load testing tool

K6 was used to run load tests. It was run 3 times for each runtime, with 10, 100, and 1000 concurrent connections within 30 seconds.

Runtimes

• Apache(prefork mode) + mod_php.
• Apache(event mode) + PHP-FPM
• Nginx + PHP-FPM
• Nginx Unit application server.
• Roadrunner
• Nginx + Roadrunner (fcgi mode)
• FrankenPHP
• FrankenPHP (worker mode)
• Swoole

All runtimes are based on official docker images. Volumes are not used, the code of the application is copied to the image during the build. PHP 8.3 is everywhere, except Nginx Unit. At the beginning of 2024 the highest PHP version supported by Nginx Unit is 8.2.

Unfortunately, I didn’t find the up-to-date versions of ReactPHP and AMPHP runtimes compatible with Symfony 7. PHPPM both GitHub and Dockerhub look abandoned.

The following PHP and Symfony settings are applied:

• php.ini-production (comes with official docker containers) is used
• Opcache enabled
• JIT enabled
• preload is configured according to Symfony best practices
• Composer autoloader is optimized
• Symfony is run in production mode
• Service container is dumped to a single file
• Symfony cache is warmed up during container build

The application code, as well as container configs, can be found on GitHub https://github.com/DimDev/php-runtimes-benchmark.

Results

1. 001_Apache+mod_php and 002_Apache + PHP-FPM showed almost the same results.
2. 003_Nginx+PHP-FPM is very close to 001_Apache+mod_php and 002_Apache + PHP-FPM when the workload is low (concurrency 10 and 100 ).
3. 003_Nginx+PHP-FPM was able to serve ~2 times more requests than Apache-based stacks when the concurrency is 1000
4. 004_Nginx Unit. First of all — WOW!!! It is ~3 times faster than the traditional and most popular 003_Nginx+PHP-FPM.
5. I don’t see a big difference between 005_Roadrunner and 006_Nginx+Roadrunner.
6. Nevertheless, Roadrunner-based stacks are more than ~2 times faster than 003_Nginx+PHP-FPM.
7. 007_FrankenPHP(non-worker mode) is pretty close to 003_Nginx+PHP-FPM from performance point of view
8. 008_FrankenPHP(worker mode). Double WOW!!! I didn’t believe from the first attempt and ran load tests 3 or 4 times. So, when sending 1000 concurrent requests FrankenPHP(worker mode) is >10 times faster than Nginx+PHP-FPM. Also, faced an issue. Couldn’t start the container with php-ini.production.
9. 009_swoole — also double WOW!. Almost the same speed as 008_FrankenPHP(wm). Extremely fast. But, keep in mind that your code should be adjusted to be run with Swoole. For Symfony we have a bundle, which extends symfony/runtime component.

Personal opinion.

1. FrankenPHP — amazing job, the first candidate to become standard de facto in PHP world.
2. Swoole — I want it to be out of the box (but disabled) PHP extension.
3. Nginx Unit — my personal choice. Despite it doesn’t support HTTP2 and many other features traditional Nginx provides, it is super easy to configure, light, and fast.

What’s next?

It is worth trying the same tests with an application connected to the database.

Try ReactPHP, AMPHP, and PHPPM-based runtimes. But before that, the appropriate packages need to be updated to work with Symfony 7.

Run containers with more resources, let’s say 2–4 CPU and 4–8GB of RAM. It would be interesting to see how well the runtimes can scale.