Automatic generation of Let’s Encrypt certificates

Expose a public endpoint over HTTPS for an application running inside your local Minikube kubernetes cluster.

5 min readJun 2, 2017

Updated 6/24

This recipe assumes a basic understanding of Kubernetes and experience with running Minikube, the local Kubernetes cluster used for development.

Overview

In this recipe you will learn how to secure and publicly expose a basic application running inside your Minikube cluster. You’ll learn to use kube-lego in your cluster to automate the generation of certificates for your applications.

The prerequisites for this exercise will be to have the following items configured and accounts already setup.

Minikube
Helm
ngrok (Pro plan required for custom domains and TLS tunnels)
Updated Nginx Ingress Controllers
A publicly registered domain name

I found the nginx ingress controller included in the minikube add-ons to be unreliable so I’m recommending upgrading these to the latest beta. You should first remove the old controllers and default backend by running $ minikube addons ingress disable

Lets get started!

1.

Install a sample application inside Minikube

Install a sample application on your local Minikube cluster. This app is from Docker Hub.

$ kubectl run hello-world --image=karthequian/helloworld:latest --port=80
$ kubectl expose deployment hello-world --type=NodePort

Make sure the service is running. The following command launches a browser and loads the application via its NodePort.

$ minikube service hello-world

Great! You can access your application within your host computer, but unfortunately you still can’t access it from another system on your LAN let alone the public network. This is because your Minikube cluster is configured though NAT (rather then a bridged network) so the IP address of your cluster isn’t known or accessible outside of your host machine.

We can address this by using a great tool called ngrok to dynamically generate a new hostname on a public domain and have it create a tunnel into our machine and forward requests onto our Minikube cluster. We’ll configure this next.

2.

Configure an ngrok reserved domain.

Ngrok terminates SSL traffic on their servers using ngrok.com certificates. We want the traffic to our application to be encrypted with our own TLS key and certificate. To do so will require setting up a TLS tunnel on a custom domain. Follow steps 1–3 in the Tunnels on custom domains section of the ngrok documentation to prepare a tunnel over your custom domain. In this tutorial we’ll be using foobar.sachin.world for our domain. Replace this value with your domain throughout the rest of this exercise.

3.

Create an Ingress resource to expose the app on the standard HTTP port on your domain

Create the following Ingress resource definition in a file named hello-world-ingress.yaml.

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: example-ingress
spec:
  rules:
  - host: foobar.sachin.world
    http:
      paths:
      - path: /
        backend:
          serviceName: hello-world
          servicePort: 80

Start the ngrok tunnel using your custom domain.

$ ngrok http -hostname=foobar.sachin.world $(minikube ip):80

Deploy the ingress resource into your Minikube cluster.

$ kubectl create -f /path/to/hello-world-ingress.yaml --save-config

You should now be able browse to http://foobar.sachin.world and see the running application.

4.

Install the certification management service

Kube-lego automates certificate management for Kubernetes Ingress resources using Let’s Encrypt. You will install a helm-chart for this into Minikube.

$ helm install stable/kube-lego --set config.LEGO_EMAIL=sppatel@gmail.com --set config.LEGO_URL="https://acme-v01.api.letsencrypt.org/directory"

5.

Update the Ingress resource for TLS

The final configuration step will be to update your ingress resource definition (hello-world-ingress.yaml) to:

Add a TLS spec for your ngrok host
Tell kube-lego to watch for changes to the ingress rule.

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: example-ingress
  annotations:
    kubernetes.io/tls-acme: 'true'
spec:
  rules:
  - host: foobar.sachin.world
    http:
      paths:
      - path: /
        backend:
          serviceName: hello-world
          servicePort: 80
  tls:
  - hosts:
    - foobar.sachin.world
    secretName: example-tls

Before we apply the changes lets tail the logs for your kube-lego pod. This way as soon as you apply the new changes to our ingress resource you’ll be able to monitor the activity thats happening within kube-lego and follow the ACME challenge to request a certificate.

$ kubectl get podsNAME READY STATUS RESTARTS AGE
hello-world-3192395452-gt68x 1/1 Running 0 6h
jazzy-otter-kube-lego-1648945840-56kz1 1/1 Running 22 8h$ kubectl log jazzy-otter-kube-lego-1648945840-56kz1 -ftime="2017-06-01T23:33:11Z" level=info msg="kube-lego 0.1.3-d425b293 starting" context=kubelego
time="2017-06-01T23:33:11Z" level=info msg="connected to kubernetes api v1.6.0" context=kubelego
time="2017-06-01T23:33:11Z" level=info msg="server listening on http://:8080/" context=acme

Finally, apply your ingress changes.

$ kubectl apply -f /path/to/hello-world-ingress.yaml

You should immediately start seeing log output from your kube-lego pod and ngrok. Kube-lego identifies the annotated ingress resource and starts the process of requesting a new certificate from Let’s Encrypt.

msg="process certificates requests for ingresses" context=kubelego
msg="creating new secret" context=secret name=example-tls 
msg="no cert associated with ingress" context="ingress_tls" msg="requesting certificate for dffce79e.ngrok.io" msg="authorization successful" 
msg="successfully got certificate: domains=[foobar.sachin.world] ...
msg="creating new secret" context=secret name=example-tls namespace=defaultHTTP Requests
-------------GET / 200 OK
GET /.well-known/acme-challenge/10H0EDxGXvJ9DIPTc85BNOyJHOS8dOiITC7SPHoo3qI 200 OK
GET /.well-known/acme-challenge/_selftest 200 OK
GET /.well-known/acme-challenge/XpdQ90uL2q6-XCTpzLZ5Z9_95B7gsrf1YLvFkJYfjvM 200 OK
GET /.well-known/acme-challenge/_selftest 200 OK
GET /.well-known/acme-challenge/RP4t1aLKp0aoJWC7-YU74OotOXBg6hQlIGP72nbRn-U 200 OK

Ingress does not always pick up changes to secrets so the nginx controller should be restarted. You can usually delete the ingress-controller pod and another new pod will come up in its place automatically due to the ingress-controler replication rule.

$ kc get pods -n kube-systemNAME                                        READY     STATUS 
...
nginx-ingress-controller-2060679912-bp9cg   1/1       Running   
...$ kc delete pod nginx-ingress-controller-2060679912-bp9cg -n kube-systempod "nginx-ingress-controller-2060679912-bp9cg" deleted

Start the TLS tunnel

At this point our application should be exposed over HTTPS and using the certificates we kube-lego generated for us. However our ngrok tunnel that is currently running is a HTTP tunnel. Terminate the tunnel and restart the tunnel as a TLS tunnel.

$ ngrok tls -hostname=foobar.sachin.world $minikube ip):443

7.

Verify Certificate

You’re done! Access your application at https://foobar.sachin.world. Verify that the certificate is for you domain and was provided by Let’s Encrypt.