Setting Up Traefik

In this post I’ll show you how to set up Traefik Proxy on a cloud compute instance.

Traefik Proxy will allow us to host multiple services under a single domain. For example, I host the following services under my personal domain:

• Photoprism (online photo album) is exposed on https://photoprism.alexishevia.com
• Navidrome (music player) is exposed on https://music.alexishevia.com
• Miniflux (RSS reader) is exposed on https://miniflux.alexishevia.com

Each of those services is running as a separate docker container inside the same compute instance.

This post assumes you already have a compute instance with Docker and Docker Compose installed. If you do not have one, check out “Setting up a Compute Instance in OCI”.

Registering your own domain

The first step in your self hosting journey is to register your own domain.

Having your own domain will allow you to easily access your services, which will all be hosted in the same compute instance.

There are many companies that sell domains (search online for “domain registrars”). I personally use Cloudflare Registrar, but feel free to use whichever one best serves your needs.

Pointing your domain to your compute instance

Go into the DNS settings for your domain, and add the following records:

| Record Type | Name                | Value               | TTL  |
| ---         | ---                 | ---                 | ---  |
| A           | <YOUR_DOMAIN.com>   | <SERVER_IP_ADDRESS> | 3600 |
| A           | *.<YOUR_DOMAIN.com> | <SERVER_IP_ADDRESS> | 3600 |

(Replace with your own domain name and the IP address for your compute instance)

Example:

| Record Type | Name              | Value         | TTL  |
| ---         | ---               | ---           | ---  |
| A           | alexishevia.com   | 143.47.51.100 | 3600 |
| A           | *.alexishevia.com | 143.47.51.100 | 3600 |

Setting up Traefik

Create a new directory on your local computer, and call it selfhost . Inside selfhost , create a file called docker-compose.yml and add the following contents:

# docker-compose.yml

version: "3.8"

services:

  traefik:
    image: traefik:v2.10.1
    restart: unless-stopped
    command:
      - --entrypoints.web.address=:80
      - --entrypoints.web.http.redirections.entryPoint.to=websecure
      - --entrypoints.web.http.redirections.entryPoint.scheme=https
      - --entrypoints.websecure.address=:443
      - --providers.docker=true
      - --providers.docker.exposedByDefault=false # require containers to define `traefik.enable=true` to be exposed
      - --api
      - --certificatesresolvers.letsencryptresolver.acme.email=${EMAIL}
      - --certificatesresolvers.letsencryptresolver.acme.storage=/acme.json
      - --certificatesresolvers.letsencryptresolver.acme.tlschallenge=true
    ports:
      - 80:80
      - 443:443
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - /var/run/docker.sock:/var/run/docker.sock:ro # allow Traefik to listen to Docker events
      - ${TRAEFIK_DIR}/acme.json:/acme.json # stores ACME (HTTPS) certificates
    labels:
      - traefik.enable=true

      # "admin" middleware definition
      # to be used by services that do not have their own security
      - traefik.http.middlewares.admin.basicauth.users=${HTTP_BASIC_USER}:${HTTP_BASIC_PWD}

      # expose the traefik dashboard
      - traefik.http.routers.traefik.entrypoints=websecure
      - traefik.http.routers.traefik.rule=Host(`traefik.${DOMAINNAME}`)
      - traefik.http.routers.traefik.service=api@internal
      - traefik.http.routers.traefik.middlewares=admin
      - traefik.http.routers.traefik.tls.certresolver=letsencryptresolver

Let’s break down this docker-compose.yml file.

version: "3.8"

services:

  traefik:
    image: traefik:v2.10.1
    restart: unless-stopped

We start by declaring we’ll be using version 3.8 of docker-compose.

Then, inside the services block we define our first service — called traefik .

The traefik service will run the traefik version 2.10.1 image from docker hub.

The restart: unless-stopped line instructs docker to restart our service if it dies for any reason— unless we manually stop it. See: Use a restart policy

command:
      - --entrypoints.web.address=:80
      - --entrypoints.web.http.redirections.entryPoint.to=websecure
      - --entrypoints.web.http.redirections.entryPoint.scheme=https
      - --entrypoints.websecure.address=:443

All entries under command are command line arguments we are sending to the traefik command on startup. Traefik docs refer to these command line arguments as the Static Configuration.

In these first 4 arguments we are defining two entrypoints: web and websecure . The web entrypoint handles http traffic over port 80 , while websecure handles https traffic over port 443 .

Because we want all traffic to happen over https, we are adding redirection rules to our web entrypoint. Any traffic coming over http will be redirected to https.

- --providers.docker=true
- --providers.docker.exposedByDefault=false

With these two lines we are enabling the Docker provider for Traefik, and we are instructing Traefik to not expose our containers by default. With these settings, we will be able to expose some docker containers (like navidrome and photoprism) to the Internet, and at the same time keep some docker containers (like databases) hidden from external traffic.

- --api

The --api argument enables the Traefik Dashboard. The Traefik dashboard provides a visual tool to see how our routes, middlewares, and services are behaving.

- --certificatesresolvers.letsencryptresolver.acme.email=${EMAIL}
- --certificatesresolvers.letsencryptresolver.acme.storage=/acme.json
- --certificatesresolvers.letsencryptresolver.acme.tlschallenge=true

The Let’s Encrypt resolver will automatically take care of registering and keeping our HTTPS certificates up to date. The certificates will be saved in the file /acme.json

Notice we’re using ${EMAIL} as a placeholder for our email address. We will create a .env file later, and we’ll define a value for EMAIL in that .env file. See: Ways to set environment variables in Compose

ports:
  - 80:80
  - 443:443

With these two rules we’re exposing ports 80 and 443 from the traefik docker container to the outside world.

volumes:
  - /etc/localtime:/etc/localtime:ro

The volumes block in docker-compose.yml allows us to map files and directories from our compute instance into the docker container.

The /etc/localtime file is the local timezone configuration file. By sharing the compute instance’s /etc/localtime with the container, we make sure the container uses the same timezone as the compute instance.

- /var/run/docker.sock:/var/run/docker.sock:ro

The /var/run/docker.sock is the Docker socket. The traefik container needs access to the Docker socket in order to listen to Docker events.

- ${TRAEFIK_DIR}/acme.json:/acme.json

In our commands block, we instructed the Let’s Encrypt resolver to store HTTPS certificates on the /acme.json file. This path is relative to the Docker container. If we don’t map this path to the compute instance, the file will be lost when the container is stopped.

With this line we are mapping the container’s /acme.json path to the compute instance’s ${TRAEFIK_DIR}/acme.json path, so it persists between container restarts.

We’ll configure a valid path for ${TRAEFIK_DIR} in our .env file later.

labels:
  - traefik.enable=true

Traefik uses docker-compose labels to set configuration values for each container.

In this case, we are enabling Traefik Proxy on the traefik container. This will allow us to expose the Traefik Dashboard through our Traefik Proxy.

- traefik.http.middlewares.admin.basicauth.users=${HTTP_BASIC_USER}:${HTTP_BASIC_PWD}

In this line we are defining a new middleware called admin, which will require any visitor to authenticate using Basic Auth.

We will use the admin middleware for any service that does not have its own built-in authentication.

We will define values for ${HTTP_BASIC_USER} and ${HTTP_BASIC_PWD} in our .env file

- traefik.http.routers.traefik.entrypoints=websecure
- traefik.http.routers.traefik.rule=Host(`traefik.${DOMAINNAME}`)
- traefik.http.routers.traefik.service=api@internal

In this block we are defining a new router called traefik. This router will receive any requests to https://traefik.${DOMAINNAME} (eg: https://traefik.alexishevia.com), and forward it to theapi@internal service— which is the Traefik Dashboard.

- traefik.http.routers.traefik.middlewares=admin

Since the Traefik Dashboard does not have any built-in authentication, we are instructing the router to use the admin middleware we defined earlier.

- traefik.http.routers.traefik.tls.certresolver=letsencryptresolver

We are also using the letsencryptresolver we defined earlier, to make sure the HTTPS certificate for this route remains valid.

Creating a .env file

Create a new file inside the selfhost directory, and name it .env

If you are going to keep this code in version control (eg: git), make sure to add .env to your .gitignore — or equivalent.

Add the following values:

# .env
EMAIL='your@email.com'
DOMAINNAME='yourdomain.com'
HTTP_BASIC_USER='your_username'
HTTP_BASIC_PWD='your_password'
TRAEFIK_DIR='/mnt/media/traefik'

Replace your@email.com with your real email, and replace yourdomain.com with your real domain.

HTTP Basic Username and Password

The HTTP_BASIC_USERand HTTP_BASIC_PWD are the username and password values you will use to protect the Traefik dashboard. However, you can’t just type in your plain password here. You need to encrypt it first.

To generate an encrypted username/password combination, you can use a site like Online Htpasswd Generator, or run the following command in a terminal:
echo $(htpasswd -nb your_username your_password)

The result will be something like:

your_username:$apr1$qtamuxse$XIWAWKgaBUG03ONJNZ3N80

Copy the portion before the colon (:) to HTTP_BASIC_USER , and the portion after the colon to HTTP_BASIC_PWD .

Eg:

HTTP_BASIC_USER='your_username'
HTTP_BASIC_PWD='$apr1$qtamuxse$XIWAWKgaBUG03ONJNZ3N80'

The acme.json file

If you take another look at our existingdocker-compose.yml, you will see we had defined the following value for our traefik service:

volumes:
  - ${TRAEFIK_DIR}/acme.json:/acme.json # stores ACME (HTTPS) certificates

With this line we are mapping the docker container’s /acme.json path to the compute instance’s ${TRAEFIK_DIR}/acme.json path, so it persists between container restarts.

And now in our .env file we are defining:

TRAEFIK_DIR='/mnt/media/traefik'

Which means we will use the /mnt/media/traefik/acme.json file in our compute instance to store our Let’s Encrypt certificates (aka ACME or HTTPS certificates).

So, let’s make sure we create this file beforehand.

SSH into your compute instance and run:

mkdir -p /mnt/media/traefik
touch /mnt/media/traefik/acme.json
chmod 600 /mnt/media/traefik/acme.json

The mkdir command creates a new directory. The touch command creates an empty file.

Since the acme.json file will hold sensitive information, we want to restrict permissions as much as possible. The chmod command lets us configure the file permissions.

If you do not set appropriate file permissions for acme.json, you will run into the following error when running your service:

The ACME resolver \"letsencryptresolver\" is skipped from the resolvers list because: unable to get ACME account: permissions 755 for /acme.json are too open, please use 600

Upload the Docker files to your compute instance

We will use the rsync utility to upload the docker-compose.yml and .env files to our compute instance.

Rsync uses the same credentials as SSH. You can run it with the following format:

rsync -av /path/to/local/dir <ssh_username>@<ip address>:/path/to/remote/dir

For example:
If I created my docker-compose.yml and .env files inside the /home/alexishevia/Projects/selfhost directory in my computer, I would run:

rsync -av /home/alexishevia/Projects/selfhost/ opc@143.47.51.100:/home/opc/selfhost/

And that will upload my files into the /home/opc/selfhost directory in my compute instance.

Running Docker and Traefik

SSH into your compute instance, then run:

cd /home/opc/selfhost
docker-compose up -d

You should see something like this:

[+] Running 5/5
 ✔ traefik 4 layers [⣿⣿⣿⣿]      0B/0B      Pulled                                                                                                                    5.7s
   ✔ c41833b44d91 Pull complete                                                                                                                                      0.9s
   ✔ 4336eee5b1b8 Pull complete                                                                                                                                      1.7s
   ✔ cb3006acbb0f Pull complete                                                                                                                                      3.8s
   ✔ f6ecd274862a Pull complete                                                                                                                                      3.9s
[+] Running 2/2
 ✔ Network selfhost_default      Created                                                                                                                             0.4s
 ✔ Container selfhost-traefik-1  Started                                                                                                                             1.2s

That means our Traefik container is up and running.

We should always take a look at the service logs, just to verify no errors were printed out. Run the following command and confirm there are no error messages printed:

docker-compose logs

You can also double check everything is good by running docker-compose ps, which should print something like this:

NAME                 IMAGE               COMMAND                  SERVICE             CREATED              STATUS              PORTS
selfhost-traefik-1   traefik:v2.10.1     "/entrypoint.sh --en…"   traefik             About a minute ago   Up About a minute   0.0.0.0:80->80/tcp, :::80->80/tcp, 0.0.0.0:443->443/tcp, :::443->443/tcp

Finally, you should be able to visit https://traefik.yourdomain.com , and see the Traefik dashboard, which should look something like this:

Take some time to look around, and get familiar with this dashboard.

You just completed the hardest part of this “Self Hosting Using Docker and a Cloud Provider” series!

In the upcoming posts, we will build upon this base setup to start hosting as many services as we want to.