How to set up Nginx reverse proxy with let’s encrypt

Why bother writing this?

The Mightywomble
Dec 30, 2018 · 8 min read

Surely this is common knowledge? you’d think so however its been pretty apparent on some of the forums I spend my time that while we know security and SSL are important, getting it set up can be a bit of a problem?

What's the scenario, why do I use this?

I don’t like web services such as Nextcloud, GitLab, Webmail being directly viewed on the internet. Even if you put an SSL certificate on each of these (which you should) and enable 2 Factor Authentication (which you should) I firstly don’t have a stack of external IP’s on to host things on port 443 and I also like to have something sat between the internet and my servers.

Rightly or wrongly, I fond the logs provided, the ability to add geolocation lockdown and other such nice to have a good thing on a network.

This is done using a reverse proxy hosted by NGINX, why NGINX? Simple its the easiest thing I’ve found to set up a reverse proxy, it's well tested, its low CPU/ram.

It’s also remembering that while a Reverse Proxy can at as your public SSL endpoint it’s not a security catch all, security is about strength in depth, putting hurdles in the way, which you are made aware of if they are knocked down.

What OS?

I will include where possible instructions for installing things on Ubuntu 18.04 and Centos 7.

Install Nginx

Nginx is a webserver, similar to Apache, I feel its a bit easier to get my head around it than Apache.

Ubuntu

Log into your Server via SSH as your user. (assumption made you can SSH into the box, otherwise, skip to the next bit if you have console access)

ssh username@hostname

Use apt-get to update your Server.

sudo apt-get update

Install nginx

sudo apt-get install nginx

By default, nginx may not start automatically, so you need to use the following command. Other valid options are “stop” and “restart”.

sudo /etc/init.d/nginx start

or

sudo service nginx start

Starting nginx: the configuration file /etc/nginx/nginx.conf syntax is ok configuration file /etc/nginx/nginx.conf test is successful nginx.

Check the service is running

sudo /etc/init.d/nginx status

or

sudo service nginx status

Centos

The process for Centos needs a little bit more setup, however its fairly similar

Log into your Server via SSH as your user. (assumption made you can SSH into the box, otherwise, skip to the next bit if you have console access)

ssh username@hostname

su to root

su -

Run command:

vi /etc/yum.repos.d/nginx.repo

Append following for CentOS 7.x:

[nginx]
name=nginx repo
baseurl=http://nginx.org/packages/mainline/centos/7/$basearch/
gpgcheck=0
enabled=1

To save and quit the file in vi press Esc followed by :x and Enter.

To install latest stable nginx server, run the following yum command:
yum install nginx

First enable nginx service by running systemctl command so that it start at server boot time:
systemctl enable nginx
Sample outputs:

Created symlink from /etc/systemd/system/multi-user.target.wants/nginx.service to /usr/lib/systemd/system/nginx.service.

Start Nginx using

systemctl start nginx

Nginx Doesn’t start..

Oh no, stop the train, time to get off.. chances are if you are running Apache on the same server that its bound to the same IP ports (80,443) that Nginx want’s to use.

try (with sudo on Ubuntu) this

netstat -plnt | grep 80

or

netstat -plnt | grep 443

You might see this

tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN 778/apache

Basically, if something is running on ports 80 or 443 you might need to change the port Nginx starts on which is out of scope for here however covered by a quick google at Tecmint: https://www.tecmint.com/change-nginx-port-in-linux/

Install letsencrypt

LetsEncrypt is your gateway to gree public facing SSL certificates, something that used to cost a few quid. We manage the requesting of certs from LetsEncrypt using a tool called certbot

Ubuntu

Certbot is in very active development, so the Certbot packages provided by Ubuntu tend to be outdated. However, the Certbot developers maintain a Ubuntu software repository with up-to-date versions, so we’ll use that repository instead.

First, add the repository:

sudo add-apt-repository ppa:certbot/certbot

You’ll need to press ENTER to accept.

Install Certbot’s Nginx package with apt:

sudo apt install python-certbot-nginx

Certbot is now ready to use, but in order for it to configure SSL for Nginx, we need to verify some of Nginx’s configuration.

Centos

The first step to using Let’s Encrypt to obtain an SSL certificate is to install the certbot software on your server. Currently, the best way to install this is through the EPEL repository.

Enable access to the EPEL repository on your server by typing:

yum install epel-release

Once the repository has been enabled, you can obtain the certbot-nginx package by typing:

yum install certbot-nginx

The certbot Let's Encrypt client is now installed and ready to use.

The Domain

For the purposes of the remainder of this tutorial we will use git.example.com as the URL we want to protect and 10.10.10.10 as the internal IP of the gitlab server we wish to provide external access to. The server on 10.10.10.10 has no SSL setup and is accessible on port 8880. We assume that the server on 10.10.10.10 has firewall access setup to allow access to port 8880

Your Environment: Considerations

Out of scope for this, however, something you may need to consider is your router, port forwarding from the internet facing router to the IP of your reverse proxy. Also, consider any firewalls you may have running.

You will also need the domain (example: git.example.com) registered and the DNS pointing to the external IP of your router.

Setup reverse proxy (Ubuntu and Centos)

The key to the reverse proxy is the configuration and the directory you can do this in is /etc/nginx/sites-enabled (technically you can use /etc/nginx/sites-available and use ln -s to symbolically link they config files as well)

cd /etc/nginx/sites-enabled

We need to create a conf file for git.example.com

vi git.example.com.conf

and add

server {
server_name git.example.com;
# The internal IP of the VM that hosts your Apache config
set $upstream 10.10.10.10:8880;

location / {
proxy_pass_header Authorization;
proxy_pass http://$upstream;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_http_version 1.1;
proxy_set_header Connection “”;
proxy_buffering off;
client_max_body_size 0;
proxy_read_timeout 36000s;
proxy_redirect off;
}

listen 80;

}

Save and exit

This is has added a basic configuration which will listen on port 80 for git.example.com and redirect the traffic to 10.10.10.10:8880

This can be checked using the command

nginx -t

This should return back

nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful

then run

systemctl reload nginx

or

service nginx reload

At this point if the firewall, router and DNS are setup right opening http://git.example.com should open up your Gitlab login.

We now need to get the http changed to https

Get the cert

Certbot is a clever bit of software and as long as your surrounding infrastructure is set up right its going to generate the certificates for you, update the config file you created to use https (443) and point to the certs, it will also redirect all http traffic to https and reload the config file for you.

This runs certbot with the --nginx plugin, using -d to specify the names we'd like the certificate to be valid for.

If this is your first time running certbot, you will be prompted to enter an email address and agree to the terms of service. After doing so, certbot will communicate with the Let's Encrypt server, then run a challenge to verify that you control the domain you're requesting a certificate for.

If that’s successful, certbot will ask how you'd like to configure your HTTPS settings:

Select your choice (option 2) then hit ENTER. The configuration will be updated, and Nginx will reload to pick up the new settings. certbot will wrap up with a message telling you the process was successful and where your certificates are stored:

Your certificates are downloaded, installed, and loaded. Try reloading your website using https:// and notice your browser's security indicator. It should represent that the site is properly secured, usually with a green lock icon.

The new conf file

If we now check /etc/nginx/sites-enabled/git.example.com.conf it will look similar to this:

server {
server_name git.example.com;
# The internal IP of the VM that hosts your Apache config
set $upstream 10.10.10.10:8880;

location / {
proxy_pass_header Authorization;
proxy_pass http://$upstream;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_http_version 1.1;
proxy_set_header Connection “”;
proxy_buffering off;
client_max_body_size 0;
proxy_read_timeout 36000s;
proxy_redirect off;
}

listen 443 ssl; # managed by Certbot
ssl_certificate /etc/letsencrypt/live/git.example.com/fullchain.pem; # managed by Certbot
ssl_certificate_key /etc/letsencrypt/live/git.example.com/privkey.pem; # managed by Certbot
include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot

}
server {
if ($host = git.example.com) {
return 301 https://$host$request_uri;
} # managed by Certbot

server_name git.example.com;

listen 80;
return 404; # managed by Certbot

}

Done, What next?

This can be repeated by using multiple conf files in /etc/nginx.sites-enabled/ pointing at a single IP Address and providing various log files which could be redirected to ELK/Graylog2 and have alerts setup based on events

This can be expanded in the Nginx config file to add Geo Location blocking or

Client authentication if you generate a Cert Pair using OpenSSL, put the client cert accessible by browser and the server cert on the Nginx then add the following code to the git.example.com.conf file under the ssl_dhparam line

# client certificate
ssl_client_certificate /etc/nginx/client_certs/root.crt;
# make verification optional, so we can display a 403 message to those
# who fail authentication
ssl_verify_client on;

Using this you can only access the service if you have the correct client certificates.

The Mightywomble

Written by

thoughts about the world I'm surrounded by, tech, politics, opinion..