Content separation
Every type of content was not created equal. Nor should it be served the same way. At a high level, you can separate your content into 3 groups:
- infinitely static — image is a perfect example of this. Once an image has been published, you should never update it (because multiple reasons) and should be able to cache it almost indefinitely.
- temporarily static — this covers the content that renders into static output. CSS, articles, etc. The content that may change in the future (requiring you to purge cache) but generally can be cached on demand for extended period of time.
- dynamic — the content that is individual for each user/load. Personalization, user profile/history, top/recent content — all these cannot be cached for any extended period of time and require re-computation or rendering on each request.
Not only those groups should be treated differently from cache strategy perspective, but they should also be treated differently when serving them from origin.
The overhead
This applies to any web server (nginx, node.js, Apache), but for the ease of example, let’s use Apache. Chances are, your httpd.conf starts with something like this (with correction to language used).
# load modules
LoadModule perl_module /opt/apache22/libexec/amd64/mod_php.so
LoadModule apreq_module /opt/apache22/libexec/amd64/mod_apreq2.so
LoadModule apreq_module /opt/apache22/libexec/amd64/mod_ssl.so# serve on port 80 for non-ssl content
Listen 0.0.0.0:80# setup mod_rewrite engine
RewriteEngine On
RewriteLogLevel 4
RewriteLog /www/logs/apache/rewrites.log# validate against rewrite checks
Include /www/etc/httpd-rewrite-global.conf# define server/thread limits
StartServers 5
ServerLimit 40
MaxClients 40
MinSpareServers 5
MaxSpareServers 10# if the file doesn't exist is static directory
# process dynamic content through the dispatcher function
RewriteCond /www/htdocs/static/$1 !-f
RewriteRule /(.*)$ /www/htdocs/dynamic/dispatcher.php [L]# if file is static and is found - serve it
RewriteRule /(.*)$ /static/$1 [L]
Most people don’t think about it, but with this setup, every request is going to load (or use) required modules and try to compare the requested URL to a (usually) long list of rewrite rules (that have a tendency to accumulate rather quickly). So the question you have to ask yourself is — does serving logo.png require loading mod_php module and does it need to be matched against dozens of legacy URL strings the business decided to keep active after the last redesign? Or, in fewer words, does every static asset need the same overhead as dynamic content? The answer is — no, no it does not.
Furthermore, is the service/thread configuration you use for your dynamic content is the same as you would need to serve your assets from origin? And again, the answer is likely no.
Separation of responsibility
Separating the serving of different types of content allows you to optimize and tune serve time for each individual type of content. The way to accomplish it is to have individual web servers running for each type of content that requires individual optimization. For example, instead of having a generic httpd.conf that listens to requests on port 80, you need to create httpd-static.conf listening on port 80 and httpd-dynamic.conf listening on different port (let’s say 8081).
httpd-static.conf
Listen 0.0.0.0:80# static content configuration parameters
StartServers 2
ServerLimit 7
ThreadLimit 1200
ThreadsPerChild 200
MaxClients 1200
MaxRequestsPerChild 0
MaxSpareThreads 1200RewriteEngine On# If it’s a local static file, go ahead and serve it,
# otherwise send to dynamic web server
RewriteCond /www/htdocs/static/%{REQUEST_URI} !-f
RewriteRule ^(.*)$ http://localhost:8081/$1 [P,L]
httpd-dynamic.conf
# load modules
LoadModule perl_module /opt/apache22/libexec/amd64/mod_php.so
LoadModule apreq_module /opt/apache22/libexec/amd64/mod_apreq2.so
LoadModule apreq_module /opt/apache22/libexec/amd64/mod_ssl.so# serve on port 8081 for non-ssl dynamic content
Listen 0.0.0.0:8081# setup mod_rewrite engine
RewriteEngine On
RewriteLogLevel 4
RewriteLog /www/logs/apache/rewrites.log# validate against rewrite checks
Include /www/etc/httpd-rewrite-global.conf# define server/thread limits
StartServers 5
ServerLimit 40
MaxClients 40
MinSpareServers 5
MaxSpareServers 10# serve content through dispatcher
RewriteRule /(.*)$ /www/htdocs/dynamic/dispatcher.php [L]
Use httpd-static.conf as a gateway, to quickly serve static content and pass on the dynamic content to httpd-dynamic.conf to do the processing.
You can configure static content to also honor individual extensions instead of files from certain directory. And in addition to having separate configuration parameters, tuned to individual types of content, this allows to easily add different logging/monitoring/caching rules to different types of content.
