Kubernetes ingress: GCE vs nginx controllers [1/3]

Damiano Giampaoli

Published in

omni:us

3 min readMar 12, 2018

In Kubernetes, the way to deal with the HTTP layer and reverse proxy rules is by using ingress.

According to the official ingress documentation, an ingress is:

An API object that manages external access to the services in a cluster, typically HTTP.

And also:

An Ingress is a collection of rules that allow inbound connections to reach the cluster services.

Unfortunately, it is still a beta feature. But since it is cool and already rather stable, it can also be considered as a viable option for production.

This is the first post of a series of 3 articles about k8s ingress, so stay tuned!

In case you are new to k8s and reverse proxy, it is recommended that you read the k8s tutorial[1] and wikipedia page for reverse proxy[2].

The Ingress configuration in kubernetes is a YAML file very similar to the reverse proxy config that you are probably writing in Apache or nginx .

Other features like IP whitelisting, requesting size limits and even Authentication & Authorization providers can be easily implemented by adding a few more line of configurations called Annotations.

With the usage of annotations, it starts to get more interesting. Just by writing and applying a few YAML lines, the entry point to your API seems to be in good shape.

But hey! Wait a minute:

Where is the documentation for the supported Annonations? and Why are there so many variants of them around?

That’s because the ingress configuration is processed by an ingress controller. Depending on the implementation of the ingress-controller, we use certain Annotations instead of others.

What’s an ingress-controller implementation?

Basically it’s a running instance, as a pod, of a http reverse proxy server. Kubernetes keeps its configuration updated, depending on the changes in the ingress configuration file. There are implementations that are using nginx, haproxy and others.

How to know which ingress-controller I’m running?

If you are using Kubernetes on a cloud provider (such as Google Cloud Platform or Amazon Web Services), which you probably already have by default, check its reference. In order to use another one you may have to deploy it yourself.

And so, how does it work on the GKE of the Google Cloud Platform?

By default, on GKE, you have an ingress-controller implemented with GCE. If you want to use another ingress controller you would have to install it. The installation process consists of deploying a pod, a service, a config map and some secrets if you need TLS support.

ngnix vs GCE ingress controllers

The default GCE ingress controller has limited functionalities. This includes the lack of support of TLS and for the whitelisting of IP’s. Furthermore, we noticed a massive delay when applying the configuration changes.

We replaced the default GCE ingress controller with nginx ingress controller, so as to to satisfy our requirements. With this, it also paved the way for the opportunities to use a lot of pre-defined annotations. It is totally worth the effort taken for the manual installation of nginx ingress controller.

Please note that the Reverse proxy syntax rules vary from one controller to another. Keep in mind that even the usage of “*” on GCE ingress differs from that of nginx ingress.

TLDR;

Ingress is a great tool but you bear the pain of using a beta software in production
It is important to carefully choose the ingress controller you want to use. Rule of thumb for GKE: use GCE if you only need reverse proxy support, nginx if you also need TLS and A&A
Additional features are implemented with Annotations. A list of Annotations and their supported controllers are here[4]. A detailed list only for nginx ingress-controller is here[5]

In the next posts we will go into further details of the GCE and nginx configurations. So, stay tuned!