Deploying Nginx Ingress and a Cert-Manager Controller on GKE using Helm 3

Photo by Joseph Barrientos on Unsplash

Our mission: we want to create a Kubernetes cluster on Google Cloud Platform using a Nginx Ingress Controller. We want to integrate with a certificate manager to automate the process of issue and renew the required certificates that we are going to deploy in our services. All the required components will be deployed using the last version of Helm.

Let’s start with Helm. Helm (https://helm.sh) is a package manager for Kubernetes. The three main concepts in Helm are:

• Chart: is a Helm package that contains all of the resource definitions necessary to run an application, tool, or service inside of a Kubernetes cluster.
• Repository: where the charts can be collected and shared.
• Release: is an instance of a chart running in a Kubernetes cluster.

At the time of writing this article, the most recent version of Helm is 3.0.1 (https://github.com/helm/helm/releases). The main changes from version 2 to 3 are:

• Removal of tiller.
• Chart repository updated.
• Chart apiVersion bumped to “v2”.
• XDG directory specification added.

If you are interested, can review all the changes in the following link: Changes since Helm 2

In order to use it on our GCP account, we are going to our Google Cloud Platform console and open the Cloud Shell. By default, your Cloud Shell comes with the Helm binary pre-installed (v2.14.1)

Our first step will be download the current binary release:

wget https://get.helm.sh/helm-v3.0.1-linux-amd64.tar.gz
tar -zxvf helm-v3.0.1-linux-amd64.tar.gz
sudo mv linux-amd64/helm /usr/local/bin/helm3

Check the version with “helm3 version”

And we are going to setup the official repository:

helm3 repo add ingress-nginx https://kubernetes.github.io/ingress-nginx
helm3 repo update

Next step, we need a Kubernetes cluster. From the Cloud Shell we are going to execute the following command:

gcloud container clusters create gke-helm3 --project "poc-helm3" --zone "europe-west1-b" --cluster-version "1.14.8-gke.12" --machine-type "n1-standard-1" --num-nodes "1" --enable-autoscaling --min-nodes "1" --max-nodes "3"

Wait until the cluster is up and running and then download the cluster credentials to our environment executing:

gcloud container clusters get-credentials gke-helm3 --zone europe-west1-b --project poc-helm3

After that, check that we can connect to our new Kubernetes cluster:

kubectl get nodes

Next move on to the next phase, setup the Nginx Ingress Controller. Our first step will be the creation of the Nginx’s namespace:

kubectl create ns nginx

And then execute the Nginx’s chart in:

helm3 install nginx ingress-nginx/ingress-nginx --namespace nginx --set rbac.create=true --set controller.publishService.enabled=true

And wait until we can see the external IP assigned to our Nginx Ingress controller:

kubectl get svc -n nginx

You can see that the controller is serving on ports 80 and 443 and the required firewall rules were setup (“VPC Networks” > “Firewall rules”):

In the next section we are going to deploy a basic application to validate our ingress controller. Our first step will be to create a deployment:

kubectl create deployment hello-app --image=gcr.io/google-samples/hello-app:1.0

And expose it (create our service)

kubectl expose deployment hello-app --port 8080 --target-port 8080

Next we are going to create an Ingress resource (hello-app-ingress.yaml)

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: ingress-resource
  annotations:
    kubernetes.io/ingress.class: nginx
spec:
  rules:
  - http: 
      paths:
      - path: /helloworld
        backend:
          serviceName: hello-app
          servicePort: 8080

Pay attention to the annotation kubernetes.io/ingress.class because we can select between:

• gce: GCP HTTP(s) Load Balancer (by default)
• nginx: GCP Network Load Balancer Nginx ingress controller

And deploy the ingress controller:

kubectl apply -f hello-app-ingress.yaml

Wait until we see the access url in “Frontends” column in the GCP Console (“Kubernetes Engine” > “Services & Ingress” and go to “Ingresses” tab):

And then we can check that our service is running clicking over the link in Frontend column:

After the validation, we can optionally reserve our assigned IP changing the type from ephemeral to static in “VPC network” > “External IP addresses” (you must reserve it if you are going to use in production)

Next, we need to setup our DNS record to point to the corresponding ip. In our case, we are going to use poc-helm3.bluekiri.tech.

In the last section, we are going to setup a certification manager. We are going to use cert-manager. Cert-manager is a native Kubernetes certificate management controller. It can help with issuing certificates from a variety of sources and it will ensure certificates are valid and up to date, and attempt to renew certificates at a configured time before their expiration date.

Let’s start making a simple change to update our domain in the ingress definition and apply the changes:

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: ingress-resource
  annotations:
    kubernetes.io/ingress.class: nginx
spec:
  rules:
  - host: poc-helm3.bluekiri.tech
    http:
      paths:
      - path: /helloworld
        backend:
          serviceName: hello-app
          servicePort: 8080

Install the CustomResourceDefinition resources separately:

kubectl apply --validate=false -f https://raw.githubusercontent.com/jetstack/cert-manager/release-0.12/deploy/manifests/00-crds.yaml

Note: don’t use — set createCustomResource=true (used in old versions), the chart will not include CRDs as part of the Helm chart (see the notes)

Create a namespace for cert-manager:

kubectl create namespace cert-manager

And add “chart.jetstack.io” to the repositories and update it:

helm3 repo add jetstack https://charts.jetstack.io
helm3 repo update

Launch the cert-manager chart:

helm install cert-manager --namespace cert-manager --version v0.12.0 jetstack/cert-manager

And check that everything is running:

Let’s create the cluster issuer (issuer.yaml):

apiVersion: cert-manager.io/v1alpha2
kind: Issuer
metadata:
  name: letsencrypt-production
spec:
  acme:
    server: https://acme-v02.api.letsencrypt.org/directory
    email: <your_email>
    privateKeySecretRef:
      name: letsencrypt-production
    solvers:
    - selector: {}
      http01:
        ingress:
          class: nginx

And create the issuer:

kubectl apply -f issuer.yaml

We are going to edit our ingress definition to add the certificate:

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: ingress-resource
  annotations:
    kubernetes.io/ingress.class: nginx
    cert-manager.io/issuer: "letsencrypt-production"
spec:
  tls:
  - hosts:
    - poc-helm3.bluekiri.tech
    secretName: poc-helm3-bluekiri-tech-tls
  rules:
  - host: poc-helm3.bluekiri.tech
    http:
      paths:
      - path: /helloworld
        backend:
          serviceName: hello-app
          servicePort: 8080

Apply the changes and after a few seconds we will see that our certificate was issued:

Last change, we are going to required that all traffic be encrypted setting the ssl-redirect annotation to true:

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: ingress-resource
  annotations:
    kubernetes.io/ingress.class: nginx
    cert-manager.io/issuer: "letsencrypt-production"
    nginx.ingress.kubernetes.io/ssl-redirect: "true"
spec:
  tls:
  - hosts:
    - poc-helm3.bluekiri.tech
    secretName: poc-helm3-bluekiri-tech-tls
  rules:
  - host: poc-helm3.bluekiri.tech
    http:
      paths:
      - path: /helloworld
        backend:
          serviceName: hello-app
          servicePort: 8080

Apply the config and that’s all. Have fun!

That is all for now but don’t miss the next episodes of the Bluekiri -team