A self-hosted ngrok

Example ngrok usage

Exposing local services can sometimes be a headache for developers - especially when you need a quick way to test your project from the outside world. Thankfully, services like ngrok exist to fix that problem by allowing developers to expose any local port to the outside world. This means that forwarding game servers over tcp, or a nodejs application over port 80 becomes a simple task.

The only drawback is that ngrok isn’t self-hosted and in order to have more than 1 ngrok process you need to pay per user per month. This can be expensive for home-labbers or startups, so luckily it’s possible to self-host ngrok!

Note: This self-hosted version of ngrok is 3 years old. Loophole Labs will be releasing https://lynk.sh soon which will be an up-to-date self-hosted tunnel for arbitrary tcp or HTTP traffic.

How? — Server

First, you’ll need a publicly accessible server. I’ll be using Digital Ocean’s smallest droplet for this, but a virtual machine on AWS, Azure, or GCP will work just as well.

Example selection for a Digital Ocean Droplet

I’ve selected a Ubuntu 18.04.3 Droplet with 1 GB RAM, 1 vCPU, and 1000 GB Bandwidth. While this is the smallest, it should be enough to run ngrok.

Once you’ve created the droplet and SSH’d into it, you need to install go. You can do this by following the following commands.

$ apt-get update && apt-get upgrade -y
$ apt-get install git make automake autoconf gcc wget -y
$ wget https://dl.google.com/go/go1.13.8.linux-amd64.tar.gz
$ tar -C /usr/local -xzf go1.13.8.linux-amd64.tar.gz
$ echo 'export PATH=$PATH:/usr/local/go/bin' >> $HOME/.profile
$ source $HOME/.profile
$ mkdir -p $HOME/go
$ git clone https://github.com/inconshreveable/ngrok.git 

These commands first upgrade all the existing packages on your Virtual Machine then install git, make, automake, autoconf, wget, and gcc which will be useful for building ngrok later. Then they download golang version 1.13.8 and install them to /usr/local/go and create the workspace $HOME/go. Finally, it clones the official ngrok repository.

Now that you’ve cloned the ngrok repository we can go ahead and make modifications to it to allow for the self-hosted version to work with SSL.

We need SSL certificates for the domain that the tunnels will be served on. For ngrok’s own service these look like <tunnel name >.ngrok.io, but for our self-hosted version we will pick our own domain.

For this tutorial I’ll be using tunnel.shivanshvij.dev. I’ll start by pointing the DNS records for *.tunnel.shivanshvij.dev and tunnel.shivanshvij.dev to my Digital Ocean Droplet by creating A records:

Next we need to install certbot so we can provision SSL certificates using Let’s Encrypt. You can use any SSL certificate for this, Let’s Encrypt just happens to be free.

Note: Loophole Labs will soon be releasing a self-hosted certificate manager for Let’s Encrypt Certificates: https://endkey.io

$ sudo apt-get install software-properties-common
$ sudo add-apt-repository universe
$ sudo add-apt-repository ppa:certbot/certbot
$ sudo apt-get update
$ sudo apt-get install certbot -y

Once certbot is installed we can provision the certificate like this:

$ certbot -d tunnel.shivanshvij.dev -d *.tunnel.shivanshvij.dev --manual --preferred-challenges dns certonly

This command tells certbot to create a certificate for two domains, tunnel.shivanshvij.dev and *.tunnel.shivanshvij.dev. Of course, you may use whatever domain you wish.

In order to verify that you control the domain you’ll be asked to create a TXT DNS record:

Please deploy a DNS TXT record under the name
_acme-challenge.tunnel.shivanshvij.dev with the following value:

667drNmQL3vX6bu8YZlgy0wKNBlCny8yrjF1lSaUndc

Once this is deployed,
Press ENTER to continue

The value of your TXT record will differ from the one shown above.

Once you’ve created this record, certbot will create your certificate in the following directory: /etc/letsencrypt/live/<your tunnel domain>/. Remember this location.

Next we will make the server binary. Make sure your working directory is the ngrok folder we cloned from GitHub and run this:

$ make release-server
$ cp bin/ngrokd /usr/local/bin

This will generate the ngrokd server binary and copy it to your /usr/local/bin folder. Now let’s create a service that will be able to start ngrokd when our machine starts:

$ cat <<EOF >> /etc/systemd/system/ngrokd.service
[Unit]
Description=ngrokd service
After=network.target
StartLimitIntervalSec=0
[Service]
Type=simple
Restart=always
RestartSec=1
User=root
ExecStart=/usr/local/bin/ngrokd -tlsKey="/etc/letsencrypt/live/<your tunnel domain>/privkey.pem" -tlsCrt="/etc/letsencrypt/live/<your tunnel domain>/fullchain.pem" -domain="<your tunnel domain>"
[Install]
WantedBy=multi-user.target
EOF

Copying that command into your SSH session will create the ngrokd.service file. Make sure you replace <your tunnel domain>.

Now simply start the service with systemctl start ngrokd and check its status using systemctl status ngrokd.

To make sure ngrokd starts on startup run systemctl enable ngrokd.

Our self-hosted ngrok server is now complete.

How? — Client

Now we need to setup the client. We can’t use the premade ngrok binaries from https://ngrok.io because their embedded SSL certificates interfere with Let’s Encrypt ones.

To fix this, we need to get the Root CA Certificate for Let’s Encrypt so that our custom ngrok client trusts Let’s Encrypt certificates. This is available from let’s encrypt: https://letsencrypt.org/certs/trustid-x3-root.pem.txt

Make sure your working directory is the ngrok folder we cloned from GitHub earlier and run the following:

$ cd assets/client/tls
$ wget https://letsencrypt.org/certs/trustid-x3-root.pem.txt
$ echo $(cat trustid-x3-root.pem.txt) > ngrokroot.crt
$ cd ../../../
$ make release-client

This will create a ngrok client in the bin folder. If you need a client for a different operating system, you’ll need to install go and compile it yourself there. I will be releasing pre-compiled Let’s Encrypt clients for Mac OS and Windows soon.

You can now move the ngrok binary in the bin folder to any machine that you will be doing development on. Once there you’ll need to create an ngrok configuration file:

$ cat <<EOF >> $HOME/.ngrok
server_addr: <your tunnel domain>:4443
trust_host_root_certs: true
EOF

This will create the ngrok configuration file $HOME/.ngrok and specify your tunnel domain.

Now you are free to run ngrok using ./ngrok 80 or ./ngrok --help

Thanks for reading! Please feel feel to contact me at shivanshvij@loopholelabs.io if you have any questions.

Check out Lynk on Product Hunt:

You can find out more about me at https://shivanshvij.com, https://github.com/shivanshvij, or https://linkedin.com/in/shivanshvij