Massively Scalable Content Caching with NGINX
for better performance
Caching mechanisms help us to increase the overall performance of an application by storing frequently accessing data at the server level. In that case, servers do not need to fetch the same set of data again and again from upstream servers. This is the same pattern that is used in Content Delivery Networks as well. CDNs allow you to maintain a cache closer to your end-users. By placing an NGINX server closer to your end-users, allows you to create your own CDN to serve application content more efficiently and effectively.
In this article, we gonna discuss how we can implement a basic caching mechanism with NGINX and how we can scale it up using appropriate NGINX configurations. So, I assume you all have some basic knowledge about NGINX and its basic configurations. If not, this 👉 article will help you to gain some.
So, this is the example scenario we gonna implement now.
We have our 1st NGINX server listening on port 8000 and it serves a simple web page to the user. We have another NGINX server listening on port 80 and it points to the 1st server via an upstream. So, all the user requests that are coming to the 2nd server or to port 80 will be pointed to the upstream then it will forward those requests to the 1st server which is listing on port 8000. In that case, again user will get the exact same web page as the response.
These are the initial configurations of the two servers.
#################### 1st Server #######################
server {
listen 8000;
root /var/www/html;
index index.html;
server_name _;
location / {
try_files $uri $uri/ =404;
}
} ################## Upstream ##########################
upstream upstream_server {
server 127.0.0.1:8000;
} ################# 2nd Server ###########################
server {
listen 80;
location / {
include proxy_params;
proxy_pass http://upstream_server;
}
}
Once we hit port 80 with the initial configurations, we will get the following set of response headers, and you can see we do not have any caching-related headers in the set.
Okay, now we are going to implement the caching mechanism in our 2nd server level which is listening on port 80. So that it will serve the cached content or the cached web page to the user without forwarding the request to the 1st server which has the actual web page.
These are the new configurations for the 2nd server.
################## Cache Zone ######################### proxy_cache_path /var/cache/nginx
keys_zone=MyCache:10m
levels=1:2
inactive=60m
max_size=20g; ################## Upstream ##########################
upstream upstream_server {
server 127.0.0.1:8000;
} ################# 2nd Server ###########################
server {
listen 80;
location / {
include proxy_params;
proxy_pass http://upstream_server;
proxy_cache MyCache;
proxy_cache_valid any 10m;
add_header X-Proxy-Cache $upstream_cache_status;
}
}
Okay, let's take newly added configuration lines one by one.
################## Cache Zone ######################### proxy_cache_path /var/cache/nginx
keys_zone=MyCache:10m
levels=1:2
inactive=60m
max_size=20g;We can define a location in the file system to store our cache and a shared memory space to store active keys with response metadata using proxy_cache_path directive. In this case, our cached responses will be stored in /var/cache/nginx, if the location does not exist NGINX will automatically create it for you. The active key name of the cache will be MyCache with 10MB of memory.
The levels parameter defines how the file structure should be created. It can have a maximum of 3 colon-separated levels. NGINX will create the cache base on the cache key which is a hashed string. Then it will store it in the provided file structure using the cache key as the file path. NGINX will break the subdirectories of the cache based on the value of the levels parameter.
The inactive parameter defines the time duration that NGINX will host the cache from its last use. After that inactive time period, NGINX will release the cache from the memory.
With the max_size directive we can specify the maximum cache size we want to keep in the configured location which is /var/cache/nginx. From that location, NGINX will load the cache keys into the shared memory space (MyCache).
Well, that’s about the proxy_cache_path directive and its parameters. Now let’s take a look at the server context.
################# 2nd Server ###########################
server {
listen 80;
location / {
include proxy_params;
proxy_pass http://upstream_server;
proxy_cache MyCache;
proxy_cache_valid any 10m;
add_header X-Proxy-Cache $upstream_cache_status;
}
}With the proxy_cache directive we have pointed our server to the shared memory space which is nothing but MyCache that we defined in the proxy_cache_path. Then in the next line, we have passed 2 arguments to the proxy_cache_valid directive to allow NGINX to cache any type of response for 10 minutes. You can replace any argument with response status codes that you want to be cached like 200, 206, 300 and so on. With the add_header directive we have added an X-Proxy-Cache header to our response headers just to check the status of the cache. Obviously, we don’t need to add this header in production applications. 😉
Okay, I hope you have understood the underlying logic or the purpose of these configurations. Now let’s hit port 80 for the first time after applying these configurations.
Here you can see now we have another response header called X-Proxy-Cache and it says it’s MISSED the cache. Yes, it’s because initially, we do not have any cache stored in the memory. But if we refresh the browser again,
It’s a HIT. It’s because now the 2nd server is serving the content or the web page from the cache, not from the 1st server. 😃
Cool, I hope you got the idea about how this NGINX caching actually works. Now I want to show you some other configurations that NGINX provides us to scale our caching mechanism.
Cache Locking
Let’s assume we don’t want to proxy requests to the upstream server (1st server) that are currently being written into the cache. In that case, we can use the proxy_cache_lock directive to instruct NGINX to write only one request into the cache and all the other subsequent requests have to wait to respond until the cache is populated.
proxy_cache_lock on;
proxy_cache_lock_age 10s;
proxy_cache_lock_timeout 3s;Here, we have specified another 2 directives. The proxy_cache_lock_age directive will define the maximum time duration that allows the first request to populate the cache. If the first request is failed to populate the cache before that time period, another subsequent request will take over the process and it will populate the cache. The proxy_cache_lock_timeout directive will fetch the response from the upstream server after the timeout but it will not populate the cache, instead, it will allow the previous request to populate the cache. We can illustrate the difference between 2 directives as,
proxy_cache_lock_age: “You are taking too long; I’ll populate the cache for you”
proxy_cache_lock_timeout: “You are taking too long for me to wait; I’m going to get what I need and let you to populate the cache in your own time”
We can use these directives inside server and location contexts to increase the performance of the cache. Default values for these 2 directives will be 5 seconds for both.
Cache Bypass
Think of a scenario that we don’t want to pull the content from the cache instead we want to get the actual response from the upstream server. In that case, we have to bypass the cache using the proxy_cache_bypass directive.
proxy_cache_bypass $http_cache_bypass;The value of this directive should be a non-zero value. In the above example, if the client wants to bypass the cache, then they need to set the cache_bypass HTTP header to any non-zero value. We can use this directive in server and location contexts.
If we want to completely turn off the cache, we can set proxy_cache off; for a given context.
Client-Side Caching
Maintaining a client-side cache will be another great way to increase the performance of an application. We can cache our static files like HTML, CSS, JavaScript in the client machine.
location ~* \.(css|js)$ {
expires 1y;
add_header Cache-Control "public";
}This location block will instruct NGINX to cache the content of CSS and JavaScript files in the client machine. The expires directive specifies the valid period of the cache, in this case, the cache will be only valid for 1 year. We have added the Cache-Control header to the HTTP response using the add_header directive with the value of public. It will allow all the caching servers to cache the resource or the web page along the way to the client’s browser. If we set it to private, only the client machine is allowed to cache the content.
With these configurations clients will not make any requests to the NGINX at all, instead, it will serve the static files from its own cache.
Hash Keys
When we want to maintain a cache specific to each user this hash key configuration approach comes in handy. 😃
proxy_cache_key "$host$request_uri $cookie_user";The proxy_cache_key directive configures the string to be hashed for the cache key and this directive can be placed in HTTP, server, and location contexts. In this example, we have instructed NGINX to cache the content based on the Host and the URI being requested, as well as a cookie that defines the user. With this configuration, we can serve user-specific cache to the end-users without serving the same cache to all the users.
Cool, in this article we have taken a deep dive into caching configurations that NGINX provides. You can also refer to the official NGINX Documentation for further information. So, I hope you have learned something from this article, and thanks for reading it till the end.
CHEERS!!!
