Useful Nginx Config for Production

// all materials present are available at

// https://github.com/seantywork/seantywork/tree/main/nginx

There are a number of benefits to having a dedicated reverse proxy that can handle load balancing over just having your web application serve all incoming requests by itself.

To mention only a few, it is much more scalable to use, it is more performance-oriented, application security can be much more easily achievable through it, user-facing functions can be summarized in a single source of truth, etc.

The common denominator for all above and beyond them is that the reverse proxy itself offers a robust, reliable, and consistent abstraction layer for some cumbersome networking heavy-liftings if otherwise handled manually by each application that sits behind it.

Why reverse proxy? That was the question I had when I first ran into the arguably most commonly used reverse proxy called Nginx.

I’m not going over all details of why it is called a reverse proxy, but the bottom line is that it is called that way because it acts like a proxy for the web applications that are behind it, and reverse because normally proxy is for egress request but in this case, it usually handles ingress request.

I know it’s hard to get a grip promptly, but anyway, that’s not the important part.

If you’re a developer and want to make your amazing application face the internet for any incoming requests, you can do it just by getting an accessible host machine (ie, AWS, GCP, Azure, and even your laptop behind a router modem) and run your app.

However, if you really want to make it not only accessible but also secure, robust, reliable, and consistent (by definition “production-ready”), you kinda need a decent reverse proxy to handle requests (or you can develop one if you’re confident it will do the job).

There are many choices available for you to go for. If you’re an AWS buff, you can go with Elastic Load Balancer, or if you’d prefer GCP to that, you have a thing called Cloud Load Balancer that acts just like the one I’m going to explain.

Nginx, as the title suggests, is the choice I would go for in most cases.

Why? I’m going to demonstrate but, in essence, it gives you some great latitude in setting up a reverse proxy + load balancer that suits various needs (which means a little more working ensues, though manageable sort IMO)

Today, I’ll take the hard way to set up a fully functioning Nginx reverse proxy and that means I’m going to place my bogus server applications and the proxy on my laptop and show how to set up a load balancer, secure connection, and Web Socket + large file handling function.

I’m going to explain how to install Nginx on a Debian/Ubuntu machine, however, let’s first step through the materials available at the Github address.

Below you can see the configuration acting as the top server configuration block called “http”. Besides that, you also have an option called “stream”, and I once dealt with it briefly when I wrote about clustered DB, but that’s not for today.

Below, you can find a virtual host configuration line, where you can have multiple rules for handling multiple upstream application requests in a single directory (or multiple directories if you think it’s a good solution).

Below, within that directory, you can see the actual server (in this case an abstracted set of functional directives) block that handles specific upstream application requests. For now, it is just listening port 80 at the host address and returning plain text “okay”.

Since there are no other server blocks available, whatever the host reference is, the request will be directed to this block.

I’m going to set up two bogus servers behind this block. One is listening on port 8000 and the other on port 8001. And they will serve the below content.

To do that, the below command will do.

Okay, to make this bogus server behind my router modem accessible from the internet, I’ll set up DHCP and port-forwarding rules, virtual machine, and proper DNS record going forward step by step.

Below, my default gateway address 192.168.0.1, but yours can surely differ.

Any decent router modem has an entry similar to the below one.

Like the below one, add a desired IP address and MAC address.

On my virtual machine console, I put in the MAC address to get provisioned the desired IP address.

If you turn on the machine, use “ip a” to check if everything is going as intended.

Now, it’s time to set up the port forwarding rules as my router model should know what to do when a request arrives on the default http port (80) and https port (443).

Now, in terms of GCP (because recently I’ve switched from AWS), this is somewhat equivalent to setting up VPC for the instance (or VM).

Now let’s set up the virtual machine. Below are installing Nginx and checking up on its directory structure.

Since the TLS methods are using vulnerable hashing functions, let’s erase them from the configuration.

Looks familiar, right?

I cloned the Github repository, and pasted the server block into the virtual host bin.

The below command will give you the result if the Nginx configuration syntax is intact.

Use

```

sudo systemctl restart nginx

```

to proceed.

Now, if I’ve done all correctly, accessing the virtual machine address on port 80 will return the plain text response “okay”.

Right.

Now you have two choices to access the 192.168.0.36 machine from the internet.

Since we’ve opened (port-forwarded) 80 and 443 ports, referring to the WAN IP of the router modem on those ports will guide your requests to the machine.

But, as the IP address is not static one, meaning it is prone to change without any notice in advance, a safer, reliable way to access it is by setting up DDNS for the router modem and registering a CNAME domain record.

Also, domain name registration is a crucial step to establish a secure connection.

I personally use Namecheap for domain registration, but anything that does the job can be a candidate.

Don’t forget that A record is for IP address, and CNAME is for another domain name.

Now, if you don’t have a DDNS for your router modem, let’s create one.

When registering domain, you can specify the host name. In my case, since my base domain is “usagecorpus.com”, if I want to add exhibit-nginx in front of it, the final domain is exhibit-nginx.usagecorpus.com

If I’ve properly registered it, I can access the machine by the domain name.

Now, let’s serve our bogus applications.

Now, it’s time to have my Nginx to load balance between the two upstream servers.

What are those proxy_set_header directives? As the name suggests, its role is to preserve (or replace in some cases) header information that is relayed to the upstream server. In “upstream” block, our bogus server addresses are present, and requests are being load balanced in least_conn manner. Default is round_robin.

Making a request to the domain name again will pop the below output from the prompt where I ran the ncat command.

Take a closer look at the first line and X-Forwarded-Proto.

As we’re adding some changes to the Nginx server block, you will see those follow suit. And also, by actually doing this yourself, you will see that the request made from your browser is waiting indefinitely for the html the server is supposed to serve because our bogus server is not properly closing the response. That will also change at the end by adding an important directive.

So far, we’ve only dealt with plain http request/response which is very insecure and not encrypted. To properly, cheaply (by cheaply in this context means “at no cost”), and easily get a certificate for https connection, you need the below step.

The certbot is a program that lets your domain get signed by Let’s Encrypt and stores the certificate in a place that can be accessed by Nginx, and even automatically updates the certificate when its expiration is due.

Running it will show you the option to which you can apply the TLS communication.

Finishing properly, you can see that the below entries have been added to the server block.

What a tremendous comfort! Otherwise, you would have to pay for a certificate authority to have your CSR signed.

Now, I’m going to redirect the http traffic to https.

It’s now a secure server block, let’s change the name.

Accessing the http:// domain, you will see that the protocol has been automatically changed to https://

Nowadays, using only http requests is rare. If you want to make a Web Socket available through reverse proxy, you need the below directives.

1.1 because Web Socket is based on http version 1.1 and should have Upgrade header to use Web Socket.

Also, if your web socket server has idle time to wait for read or write, add the first two of the below directives and set the value properly so that the connection doesn’t get terminated abruptly.

Now, the last two are usually related to handling the uploading of large files in association with client_max_body_size directive, it also has some special function in this particular example.

Do you remember I said our bogus server doesn’t properly close the response so that the request made from the browser hangs indefinitely?

Let’s see what happens after adding the last directives.

Voila! With the directives, Nginx doesn’t wait for the response to close, it relays whatever it receives from the upstream.

Because of this feature, the directives are important when you have to get a real-time progress when some large (20G<) is being uploaded, otherwise without them, server response will be “waited” until the whole upload is finished.

And also, there is another thing to be careful of.

Look at the below one.

When uploading a huge file, make it ip_hash since it guarantees that the data being sent to the upstream always sticks with the upstream that it first got directed to. Using least_conn? You will see some mysterious log where, in the middle of uploading, data is switching uncontrollably between upstreams.

There are more I wish to cover, but let’s make it next time since it has been an already gruesomely long article.

Thanks!