Exposing GKE applications leveraging the built-in ingress

Today I decided to play a bit with GKE and see how to expose a simple demo application through the default GKE ingress.

To me, what exposing an application in a production environment means is (at the very minimum…):

• Exposing my application on HTTPS only
• Redirect any HTTP connection to HTTPS automatically
• Use a static IP, rather than an ephemeral IP
• Having Google automatically managing certificates for me

While I’m used to go through Google Cloud documentation, I realized it still took me quite some time to put all pieces together, and so I decided to write this short tutorial.

Where I started from

All I had was a minimal setup, which I usually keep active to try out stuff or to run demos. Nothing complex:

• A GKE cluster (I use a private one, but public clusters would be ok as well)
• A public domain name. For the sake of this exercise we’ll call it mydomain.com.
• A management client (my laptop!), configured to access the cluster control plane using kubectl and gcloud already setup.

Let’s deploy!

This will be a “quick and dirty” walkthrough. I will stress few things, as we’ll move through the deployment and put some relevant links at the end of the article, so you can deep dive in most of the steps.

For this exercise, we’ll try to expose a simple web application at https://www.mydomain.com.

First, let’s deploy a sample nginx application

# app.yaml
apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: null
  labels:
    app: ingress-test
  name: ingress-test
spec:
  containers:
  - name: ingress-test
    image: nginx:latest
    imagePullPolicy: Always
  dnsPolicy: ClusterFirst
  restartPolicy: Always
status: {}
---
apiVersion: v1
kind: Service
metadata:
  creationTimestamp: null
  labels:
    app: ingress-test
  name: ingress-test-svc
  annotations:
    cloud.google.com/neg: '{"ingress": true}'
spec:
  ports:
  - name: ingress-test-port
    port: 80
    protocol: TCP
    targetPort: 80
  selector:
    app: ingress-test
  type: ClusterIP
status:
  loadBalancer: {}

Notice the annotation cloud.google.com/neg: ‘{“ingress”: true}’ for the service ingress-test-svc. This has two benefits:

• The Global Load Balancer (GLB) allocated as a consequence of the service creation will load balance traffic directly between the pods, rather than the GKE nodes, thus leveraging the so called container-native load balancing, so Network Endpoint Groups (NEGs). This will avoid a second level of load balancing / traffic routing inside the Kubernetes cluster, in favor of a more reliable and faster routing.
• You’ll be able to reference ClusterIP services in your ingress, rather than having to create NodePort or LoadBalancer services.

Let’s deploy the application:

kubectl apply -f app.yaml

Let’s allocate a static IP. For the demo, we’ll do this quickly with gcloud, but Terraform or the GUI could be used as well. From the cloud shell, let’s run

gcloud compute addresses create ingress-test-ip --global

The IP address is immediately visible with

gcloud compute addresses describe ingress-test-ip --global
address: X.X.X.X
addressType: EXTERNAL
creationTimestamp: '2021-12-23T01:32:03.047-08:00'
description: ''
id: 'xxxxxxxxxxxxxxx'
ipVersion: IPV4
kind: compute#address
name: ingress-test
networkTier: PREMIUM
selfLink: https://www.googleapis.com/compute/v1/projects/xxxxxxxx/global/addresses/ingress-test-ip
status: IN_USE

Next, we need to create an A DNS record (in my case, I did this in Cloud DNS, since my domain is managed there), so that our FQDN www.mydomain.com points to the GLB IP address.

Let’s now create the certificate that will be exposed by the load balancer. While this could be created with Terraform or through the GUI, there’s a convenient way to request one from GKE directly:

# cert.yaml
apiVersion: networking.gke.io/v1
kind: ManagedCertificate
metadata:
  name: ingress-test-cert
spec:
  domains:
    - www.mydomain.com

Substitute the domain with the one you own, and apply:

kubectl apply -f cert.yaml

A new certificate will be requested, but it won’t be active until verified, so until the load balancer gets created and a DNS entry will point to it (this last one is already true in our case). We’ll be able to periodically check the status of the certificate from the GUI or directly through kubectl.

kubectl get managedcertificates
NAME                AGE     STATUS
ingress-test-cert   2m      Pending

Let’s now create the ingress object:

# ingress.yaml
apiVersion: networking.gke.io/v1beta1
kind: FrontendConfig
metadata:
 name: ingress-test-fe-config
spec:
 redirectToHttps:
   enabled: true
   responseCodeName: PERMANENT_REDIRECT
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: test-ingress
  annotations:
    kubernetes.io/ingress.class: "gce"
    kubernetes.io/ingress.global-static-ip-name: "ingress-test-ip"
    networking.gke.io/managed-certificates: "ingress-test-cert"
    networking.gke.io/v1beta1.FrontendConfig: "ingress-test-fe-config"
spec:
  defaultBackend:
    service:
      name: ingress-test-svc
      port:
        number: 80

In this file, we create two objects:

• A frontend configuration that redirects any HTTP connection to HTTPS (this will be automatically translated into a GLB configuration). Thanks Bernhard Weisshuhn!
• An ingress object, also referencing the configuration.

Speaking of annotations:

• The ingress.class is set to gce. I like to specify this explicitly, although that’s the default value when one is not specified manually.
• We tell the ingress to use the static IP we previously created, using its GCP resource name.
• We reference the managed certificate object we just requested in GCP through Kubernetes.
• We reference the FrontendConfig object, just created in the same file, above.

Let’s apply:

kubectl apply -f ingress.yaml

The load balancer creation will take some time. During the provisioning, we should see something like

kubectl get ingress
NAME           CLASS    HOSTS   ADDRESS   PORTS   AGE
test-ingress   <none>   *                 80      4s

But after a few minutes, when the load balancer is ready, we should see the GLB IP showing up in the output (at least, that’s usually a good sign!)

NAME           CLASS    HOSTS   ADDRESS         PORTS   AGE
test-ingress   <none>   *       x.x.x.x.        80      158m

The certificate provisioning might take longer (Google says it might take up to 60 minutes before the certificates might get validated). For me, it took no more than 5 minutes. This is what you should see:

kubectl get managedcertificates
NAME                AGE     STATUS
ingress-test-cert   15m      Active

There’s nothing else remaining then opening a browser and connect to https://www.mydomain.com, which -hopefully- will show the default nginx page.

Notice that when we try to connect to http://www.mydomain.com, we’ll always be redirected to its https version.

Useful links

Here are some useful links I used while going through the exercise. They contain more variants (i.e. self-managed certificates) and should be definitely used to consolidate the concepts explored in this article.

• Using multiple SSL certificates in HTTPS load balancing with Ingress
• Using Google managed certificates
• Configure ingress for external load balancing
• Container-native load balancing through ingress

Hope this short tutorial has been useful and helped you to save time, or at least to arouse your curiosity.

More articles coming soon. Stay tuned!

Thanks Simone Ruffilli for reviewing!