Apache2 and php fpm performance optimization — Step-by-step guide

I had to handle high traffic loads in my career and I fought with down-time, not used memory and a lot of trouble in my past. In this article I want to give a step-by-step guide to apache2 performance settings, which is a concentrated result of a lot of reading and trying.

My environment

Precalculation of average memory usage and maxclients/max-children

1. Calculate process size

You need to know how many processes can run on your machine. So calculate the process size of your main CPU/memory drivers is necessary.

There are several calculation methods out there. Personally I prefer this python script as it takes shared memory into account and will give insights into real memory usage.

cd ~
wget https://raw.githubusercontent.com/pixelb/ps_mem/master/ps_mem.py
chmod a+x ps_mem.py
sudo python ps_mem.py

ps_mem.py will produce output like this:

Here you can see that there are 30 apache2 processes, consuming a total of 139MiB, so each Apache process is using roughly 5MiB of RAM. The php-fpm5.6 process will use about 50MiB.

2. Calculate apache MaxRequestWorkers

To be safe though, I’ll reserve 15% of memory for all other processes (in my case ~2,4GiB) and round up apache process size to 5MiB.

MaxRequestWorkers = (Total RAM - Memory used for Linux, DB, etc.) / process size
MaxRequestWorkers = (16384MB - 2400MB) / 5MB = 2800

3. Calculate php-fpm max-children

To be safe though, I’ll reserve 1 GiB for all other processes and round up php process size to 55MiB.

maxclients = (Total RAM - Memory used for Linux, DB, etc.) / process size
maxclients = (16384MB - 2400MB) / 55MB = 256

My colleague Thomas Herweg prepared this Excel Sheet for the calculation: https://s3.buckpesch.io/downloads/apache_performance.xlsx

Detailed Setup

In the /etc/apache2/mods-enabled/mpm-event.conf or /etc/apache2/mods-enabled/mpm-worker.conf file:

<IfModule mpm_*_module>
ServerLimit (Total RAM - Memory used for Linux, DB, etc.) / process size
StartServers (Number of Cores)
MinSpareThreads 25
MaxSpareThreads 75
ThreadLimit 64
ThreadsPerChild 25
MaxRequestWorkers (Total RAM - Memory used for Linux, DB, etc.) / process size
MaxConnectionsPerChild 1000
</IfModule>

Note that the default settings did not contain the “ServerLimit”, so I added it here.

In the /etc/php/7.1/fpm/pool.d/www.conf change the following settings:

pm = dynamic            
pm.max_children (total RAM - (DB etc) / process size)
pm.start_servers (cpu cores * 4)
pm.min_spare_servers (cpu cores * 2)
pm.max_spare_servers (cpu cores * 4)
pm.max_requests 1000

To learn about all php-fpm settings I recommend watching the video on that page: https://serversforhackers.com/c/php-fpm-process-management

My final settings

My server has 16GB RAM and 4 CPUs à 2,4GHz. My avarage apache process has 5MB, an average PHP process takes 55MB.

/etc/apache2/mods-available/mpm_event.conf

# Optimized settings for avg. apache process 15MB and AWS EC2 m4.xlarge Server
<IfModule mpm_event_module>
ServerLimit 2800
StartServers 4
MinSpareThreads 25
MaxSpareThreads 75
ThreadLimit 64
ThreadsPerChild 25
MaxRequestWorkers 2800
MaxConnectionsPerChild 1000
</IfModule>

/etc/php/7.1/fpm/pool.d/www.conf

; Optimized for php-fpm request size of 55MB on AWS EC2 m4.xlarge (4CPU cores, 16GB RAM)
pm = dynamic
pm.max_children = 256
pm.start_servers = 20
pm.min_spare_servers = 10
pm.max_spare_servers = 20
pm.max_requests = 1000

Save your settings and restart your apache and php-fpm processes

sudo service apache2 restart
sudo service php7.1-fpm restart

Test you settings

To test your server settings you can run Apache Bench tests and see how your server behaves in htop .

Open 2 terminals and the following command to test 5000 requests with a concurrency of 100 parallel requests:

ab -n 5000 -c 100
Load test using apache bench

Simulation process improvement (proposed by zzzplayer)

One suggestion I can make is to also tell the readers to run “ps_mem” before and *quickly after* the simulated Apache bench (*quickly after* because the process will remove idle child processes). This will display the exact number of child processes before and after the simulated workload for both Apache and PHP-FPM.
“htop” doesn’t display such exact number of child processes from both Apache and PHP-FPM.
Running both “ps_mem” and “htop” is fine, but runnig “ps_mem” is more important because the point of the article is to make sure the child processes of both Apache and PHP-FPM increase as the workload increase.

I hope this helps. Drop me a line, when you have other experience or think I might can improve my formular/calculation. As well I might create a simple web-interface to calculate the settings… But now I have to go back to work ;-)

References