Setup Traefik v2 for HA on Kubernetes

Yitaek Hwang
Aug 5 · 4 min read

Using cert-manager to manage Let’s Encrypt TLS certs and running multiple replicas of Traefik v2.

Traefik and Kubernetes Logo
Traefik and Kubernetes Logo

In my previous post, “Quickstart with Traefik v2 on Kubernetes,” I went over a quick 5-minute end-to-end setup of Traefik, Let’s Encrypt, and Cloudflare to handle HTTPS requests on Kubernetes. While that setup with Traefik CRDs is convenient for automatically creating and renewing certs via IngressRoute definitions, it runs with a single instance of Traefik, meaning that it is not highly available. In other words, Traefik becomes the single point of failure for all ingress traffic to your cluster.

In Traefik v1, there was beta support for clustering / HA mode using a KV store (e.g. Consul, etcd, etc). However, Traefik v2 removed support for storing ACME/Let’s Encrypt certificates in a KV store, citing bugs with the raft consensus algorithm (#4851, #3487, #5047, #3833). Automatic cert management feature moved to TraefikEE, leaving open-source users to either run a non-HA version or implement a custom solution to certificate management.

Traefik documentation recommends using cert-manager as the Certificate Controller and notes limited support for the Ingress Route CRD:

When using the Traefik Kubernetes CRD Provider, unfortunately Cert-Manager cannot interface directly with the CRDs yet, but this is being worked on by our team. A workaround is to enable the Kubernetes Ingress provider to allow Cert-Manager to create ingress objects to complete the challenges. Please note that this still requires manual intervention to create the certificates through Cert-Manager, but once created, Cert-Manager will keep the certificate renewed.

This post walks through how to get around this limitation and run Traefik v2 in HA mode on Kubernetes. I will be using Cloudflare as my DNS provider and ACME challenge solver, but feel free to use any other Let’s Encrypt supported providers.

All of the code is also available on Github:

Prerequisites

  • Kubernetes Cluster (e.g. GKE)
  • Helm v3
  • DNS provider (e.g. Cloudflare)

Install Traefik

We will deploy Traefik to traefik namespace:

Now let’s deploy Traefik with 3 replicas. You can see the values in traefik/traefik-values.yaml :

Helm Values for Traefik HA Setup

Wait for the deployments to come up and make note of the Load Balancer IP.

Install Cert-Manager

Cert-manager is an open-source tool to automate the issuance and renewal of TLS certificates:

Image for post
Image for post
cert-manager diagram — Image Credit: cert-manager documentation

We will install it in the namespace cert-manager:

Add the Jetstack Helm repo and install CRDs:

Wait for all the cert-manager pods to come up:

Deploy an Application

For the sake of the demo, we will deploy the whoami app in the default namespace (see under whoami directory for deployment, service, and ingress files). You can replace this with your application or well-known Helm chart (e.g. Grafana, Kibana, etc).

whoami default deployment, service, ingress routes

Replace whoami.example.com with your FQDN and deploy:

Create Certificates

In order to issue new certificates, we need to first define an Issuer. In this example, I’ll be using Cloudflare for ACME Issuer type, using Let’s Encrypt’s staging server. You can also find other supported configurations (SelfSigned, CA, Vault, Venafi, and External Issuer Types) on the documentation.

cert-manager issuer example

Configure the email and solvers sections in certs/issuer.yaml. To use Cloudflare as DNS01 challenge solver, first create a new API token with the following settings:

Permissions:

  • Zone - DNS - Edit
  • Zone - Zone - Read

Zone Resources:

  • Include - All Zones

Mount the token as a Kubernetes secret:

Finally, configure the certificate (modify the commonName, secretName, and dnsNames as needed in certs/whoami-cert.yaml) and deploy:

whoami application example cert

Set Up DNS

Check if the certificate has been generated:

You can also look at Traefik’s debug logs to watch the cert become active.

Finally, point the DNS record to the IP address of the Load Balancer to see a TLS enabled site backed by HA Traefik + cert-manager. Optionally, you can deploy the HTTPS redirect middleware for completeness.

Now we have a HA deployment of Traefik on Kubernetes. The downside to using cert-manager is that the user must now remember to create the cert before deploying the IngressRoute, but achieving HA is more important in production to avoid downtime.

Dev Genius

Coding, Tutorials, News, UX, UI and much more related to development

Sign up for Best Stories

By Dev Genius

The best stories sent monthly to your email. Take a look

By signing up, you will create a Medium account if you don’t already have one. Review our Privacy Policy for more information about our privacy practices.

Check your inbox
Medium sent you an email at to complete your subscription.

Yitaek Hwang

Written by

Sr. Software Engineer at Axoni writing about cloud, DevOps, and SRE topics: https://yitaekhwang.com

Dev Genius

Coding, Tutorials, News, UX, UI and much more related to development

Yitaek Hwang

Written by

Sr. Software Engineer at Axoni writing about cloud, DevOps, and SRE topics: https://yitaekhwang.com

Dev Genius

Coding, Tutorials, News, UX, UI and much more related to development

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app