NGINX With ModSecurity and Brotli: Production setup (Dockerized)

Vijay Nallagatla
Aug 16, 2019 · 5 min read

Have you ever heard of Load Balancer? Reverse Proxy?
If you’ve ever heard the term ReverseProxy or Load Balancer being thrown around and wondered to yourself what that term meant or how to use NGINX as Reverse Proxy, this article aims to alleviate those concerns and develop a basic understanding. Setup a production grade Customized NGINX Docker Image with ModSecurity and Google’s, Brotli Lossless file compression.

NGINX (Engine-x) ?
Load balancing is an excellent way to scale out your application and increase its performance and redundancy. Nginx, which is a popular web server software, can be configured as a simple yet powerful load balancer to improve your servers resource availability and efficiency. Nginx is one of a handful of servers written to address the C10K problem. Unlike traditional servers, Nginx doesn’t rely on threads to handle requests. Instead, it uses a much more scalable event-driven (asynchronous) architecture. In a load balancing configuration, NGINX acts as a single entry point to a distributed web application working on multiple separate servers. In a Microservice Architecture, NGINX helps serve services running on multiple servers.

In My organisation, I was asked to serve Multiple Web Applications (SPA’s) and Backend API’s using only LoadBalancer/ReverseProxy being exposed to internet. We are running light weight services over a dockerized environment with docker-compose and hence we chose NGINX as a reverse proxy and used ModSecurity to handle 97% of known security vulnerabilities.

Since we are into Microservice Environment, we have developed an API Gateway for all the internal microservices communication through docker bridge network and exposed Gateway to NGINX. We are going to eliminate API Gateway and include the reverse proxy conf with NGINX itself to route each requests from client to respective services, and of course each request from client will be authenticated using Authentication service using OAuth. In this way we can scale the backend services with replicas on a same host or on multiple hosts and NGINX will be configured in a way that it can route the requests to services running on hosts based on the number of requests.

ModSecurity
“Web applications — yours, mine, everyone’s — are terribly insecure on average. We struggle to keep up with the security issues and need any help we can get to secure them.” – Ivan Ristić, creator of ModSecurity

OSS version of NGINX does not handle security issues and it’s included as part of NGINX plus. Since NGINX WAF uses open sourced ModSecurity to handle most of the known security breaches and vulnerabilities, We have built a customized Docker image to include ModSecurity.
ModSecurity is an open source, cross-platform web application firewall (WAF) module. Known as the “Swiss Army Knife” of WAFs, it enables web application defenders to gain visibility into HTTP(S) traffic and provides a power rules language and API to implement advanced protections.

Google/Brotli
All the Web browsers support file encoding and for several years gzip had become the primary encoding type until google introduced another lossless compression format which is supported by most of the modern web browsers.
Brotli is a generic-purpose lossless compression algorithm that compresses data using a combination of a modern variant of the LZ77 algorithm, Huffman coding and 2nd order context modeling, with a compression ratio comparable to the best currently available general-purpose compression methods. It is similar in speed with deflate but offers more dense compression. The specification of the Brotli Compressed Data Format is defined in RFC 7932.
Brotli is open-sourced under the MIT License.

As part of production Setup and Performance improvements we have integrated following changes into nginx conf.

  1. All the API’s req/res are secured through IAM — Authorization and authentication.
  2. NGINX as reverse proxy and load balancer for all the requests
  3. Protocol change from HTTP/1.1 to HTTP/2
  4. Encoding compression algorithm: upgradation from gzip to Brotli (introduced and used in Google’s infrastructure) reduced the content compression over the network for 80% and above compression (ie. 10MB over the network compressible to ~1.5MB)
  5. Browser Cache: Cache the static files completely in browser for 365 days so the UI will be loading from browser’s in-memory/disk cache.
  6. Rate Limiting: We can rate limit the API’s req/res to stop DDOS attack which makes our server’s to block such requests and improve HA (High availability). We are allowing 30 req/res for unique client IP’s.
  7. Content type limiting
  8. Security headers to avoid cross platform attacks through scripts (X-Frame-Options, X-XSS-Protection, X-Content-Type-Options, Referrer-Policy, Content-Security-Policy, Strict-Transport-Security)
  9. Strict SSL Policies
  10. Efficient use of Buffers
  11. ModSecurity to handle 97% of Known Cyber attacks, suspicious requests, cross origin requests, anomaly detection etc..,
  12. HTTP request timeouts to handle stalled requests
  13. Header Content size limitation for handling bulk upload attacks
  14. Black List and Whitelisting known and unknown source requests
  15. Control Allowed Request headers
  16. Improve TTFB — TIME to FIRST BYTE for serving static files by caching files into temporary folder /tmp.

NGINX Load Balancer configuration

http {
upstream backend {
least_conn
server srv1.example.com;
server srv2.example.com;
server srv3.example.com;
server 10.0.0.123;
server 192.168.0.3;
}
server {
listen 80;
location / {
proxy_pass http://backend;
}
}
}

NGINX By default follows Round-Robin algorithm to route the requests to configured upstream servers unless we explicitly mention one of the following load balancing conf:
Least connected load balancing: With the least-connected load balancing, nginx will try not to overload a busy application server with excessive requests, distributing the new requests to a less busy server instead.
Session persistence: If there is the need to tie a client to a particular application server — in other words, make the client’s session “sticky” or “persistent” in terms of always trying to select a particular server — the ip-hash load balancing mechanism can be used.
Weighted load balancing: When the weight parameter is specified for a server, the weight is accounted as part of the load balancing decision.

Hardening Security

Creating Docker Image to include ModSecurity:3.0.0 and Brotli


NGINX Production Setup


The Startup

Medium's largest active publication, followed by +567K people. Follow to join our community.

Vijay Nallagatla

Written by

The Startup

Medium's largest active publication, followed by +567K people. Follow to join our community.

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade