Kubernetes Base Config for nginx-ingress + cert manager + Helm/Tiller

We recently updated our kubernetes cluster to migrate from kube-lego (since it’s now deprecated), as well as introduced Role Based Access Controls (RBAC) to our services.

The following article outlines how to install cert manager and nginx-ingress using Helm/Tiller. We use a Git repository to store all our configuration YAMLs, so if you’re used to deploying through the CLI, you can still get the details you need out of these YAML files.

Set up Helm/Tiller with RBAC

Helm is essentially a package manager for kubernetes, and Tiller is its corresponding server-side piece. You install Helm locally and initialize it within your current kubectl context and it will install tiller on your cluster.

Start by installing Helm on your computer:

# Mac
brew install kubernetes-helm
# From Scripts
$ curl https://raw.githubusercontent.com/kubernetes/helm/master/scripts/get > get_helm.sh
$ chmod 700 get_helm.sh
$ ./get_helm.sh

Before you initialize Helm, make sure to set up the RBAC role for Tiller. Below is the service account configuration you can use.

We set up our Tiller service account configuration as a ClusterRole since we’re not running a massive cluster and we’re not that concerned with namespace specific permissions for certificate management. You can also use role binding and specify a namespace if you want that level of granularity.

Apply this file with:

kubectl apply -f ./tiller.serviceaccount.rbac.ymal

With Tiller’s permissions in place on the cluster, we can initialize Helm locally. If you have multiple contexts you’re managing, like we do, make sure that you are on the context you want to install Tiller on:

kubectl config current-context

If you’re on the right context, initialize Helm with RBAC for Tiller:

helm init --service-account tiller

Once you get the confirmation, you’re done. Check your pods to confirm:

kubectl get pods --all-namespaces
>> kube-system tiller-deploy-75f5797b-rfcwr 1/1  Running 0 1m

Set up cert manager

Cert manager allows kubernetes to provision TLS certificates for your various services. You can find some pretty helpful Getting Started guides on cert managers GitHub page.

One thing to make sure you have in place before, to save time, is to decide whether you want to use a name spaced issuer or cluster issuer. Again, we don’t have a massive cluster, so using a cluster issuer would mean just configuring one issuer to issue certificates for all our services. You can read about the difference’s in cert managers reference docs here.

Below is our cluster issuer configuration:

Set your private key ref accordingly; you’ll need to use that name in the next Helm command.

With these two pieces in place, you can do a one command install of cert manager with Helm:

helm install \
--name cert-manager \
--namespace kube-system \
stable/cert-manager \
--set ingressShim.defaultIssuerName=letsencrypt-prod \
--set ingressShim.defaultIssuerKind=ClusterIssuer

If you’ve already installed cert manager and just need to add the above two configurations, you can run this command:

helm upgrade cert-manager \
stable/cert-manager \
--namespace kube-system \
--set ingressShim.defaultIssuerName=letsencrypt-prod \
--set ingressShim.defaultIssuerKind=ClusterIssuer

These two lines are key if you’re upgrading from kube-lego:

--set ingressShim.defaultIssuerName=letsencrypt-prod \
--set ingressShim.defaultIssuerKind=ClusterIssuer

They allow you to utilize the same annotations in your ingress as kube-lego does to request TLS — that’s these lines here (you’ll also see them at the end in the ingress configuration):

kubernetes.io/tls-acme: 'true'
nginx.ingress.kubernetes.io/tls-acme: 'true'

Check all is running with the following command:

kubectl get pods --all-namespace

Hopefully you see this somewhere in there:

kube-system cert-manager-cert-manager-bcb987f-2ggf2 1/1 Running 0 1m

Set up nginx-ingress with Helm & RBAC

The great thing about nginx-ingress controller is that all the files you need to configure are available in their Git repo here. But we’re using Helm, so let’s accept the darkness of the black box and just run this command instead:

helm install stable/nginx-ingress --name nginx-ingress --set rbac.create=true

And that’s it… actually… just call it a day … it’s up. The only thing you might want to add is a configmap.yml for some custom nginx settings:

Before you can reach your services, make sure to point your main DNS endpoint to your shiny new nginx load balancer.

In route 53, select your name record and set an ALIAS to the ELB load balancer that kubernetes just started up:

Set up a service and deployment

You can read up on a sample WordPress deployment we did here to see an example for these configurations. All you need is to get an ingress up and running you’re on SSL. Woo!