Ingress in Kubernetes with NGINX

Ingress Controller and Ingress, the difference:

Lot of people are confused between Ingress and Ingress Controller so, let us have a high level clarification on what they are, to better understand the further scenarios in the blog.

Ingress is a definition of how the traffic from outside the cluster can reach the services running inside the cluster. It is like a configuration request and this request is implemented by the Ingress Controller. Ingress Controller basically looks for Ingress resources and implements all the traffic routes.

Let us look at some possible scenarios in a single Kubernetes cluster. We are using nginx ingress controller in this blog for these scenarios:

• One Ingress controller with multiple Ingress resources
• Multiple Ingress Controllers
• One host with Ingress configuration spread across namespaces

One Ingress controller with multiple Ingress resources:

The following diagram shows high level flow of traffic in this scenario where there is one Ingress Controller in its own namespace ‘controller_ns’ and 2 Ingress resources in different namespaces ‘host1_ns’ and ‘host2_ns’. Both the Ingress resources are routing traffic to different services configured as backends and the traffic reaches the pods through these services.

How is this achieved?

The default Ingress class of NGINX Ingress Controller is ‘nginx’, which means that it only handles configuration resources with the Ingress class set to ‘nginx’. So, Once the Ingress controller is deployed, we can use its Ingress class name in the annotations in Ingress resource’s spec file as follows:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: myingress
  namespace: host1_ns
  annotations:
    kubernetes.io/ingress.class: nginx

The above annotation will make this Ingress resource visible to the Ingress Controller and the Ingress Controller can read the configuration and implement it.

Multiple Ingress Controllers:

One might think that the above scenario is the most common one but when the cluster is deployed using a cloud provider then chances are that by default might include the Ingress Controller for the HTTP load balancer of the cloud provider, and you want to use NGINX Ingress Controller which brings us to this scenario “Multiple Ingress Controllers” in a single Kubernetes cluster.

There are few ways we can configure multiple ingress controllers:

Ingress Class Name:
The Ingress Controller will look at only the Ingress resources that are configured with the annotation of its ingress class name. You can customize the class through the ‘-ingress-class’ command-line argument or by using ‘ingressClass’ argument under the controller section in the values.yaml file of the Nginx Ingress Controller helm chart as follows:

controller1:
  ingressClass: "nginx-controller1"
  # Other configuration values for controller1

controller2:
  ingressClass: "nginx-controller2"
  # Other configuration values for controller2

Watch Namespace:
We can configure Ingress Controller to handle configurations only from selected namespaces. We can select namespaces using ‘-watch-namespace’ command-line argument or it can be added in the deployment YAML file as follows:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-ingress-controller
  namespace: ingress-controller-namespace
spec:
  selector:
    matchLabels:
      app: nginx-ingress
  template:
    metadata:
      labels:
        app: nginx-ingress
    spec:
      containers:
      - name: nginx-ingress-controller
        image: nginx/nginx-ingress:latest
        args:
        - /nginx-ingress-controller
        - --ingress-class=nginx
        - --watch-namespace=your-specific-namespace  # Specify the namespace to watch
        # Other arguments and configurations for the NGINX Ingress Controller

Both the options above can be used in conjunction with each other depending on the requirement.

One host with Ingress configuration spread across namespaces:

Let us take a scenario where you have one host ‘mypc.com’ and there are multiple paths like ‘/accessories’, ‘/software’, ‘/hardware’ etc then, we have separate namespaces for accessories, software and hardware and as there is a single host, there can only be one Ingress resource. How can we configure this when the paths are spread across namespaces? We can use Cross-namespace configuration, in which Ingress resource is split into one master and multiple minions and these minions can be in different namespaces. This enables easier management when using a large number of paths.

Some points to understand:

• There can only be one master and Ingress resource can be identified as master using the annotation ‘nginx.org/mergeable-ingress-type: “master”’. Paths should not be mentioned in the master’s spec file.
• There can be multiple minions in the same or different namespaces. Ingress resource can be identified as minion using the annotation ‘nginx.org/mergeable-ingress-type: “minion”’. It is best to avoid conflicting paths in the minions but, if there is a conflicting path then, the path defined on the oldest minion will be used.

Let us take a look at the spec files for reference:

Master:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: mypc-ingress-master
  namespace: ingress_ns
  annotations:
    nginx.org/mergeable-ingress-type: "master"
spec:
  ingressClassName: nginx
  tls:
  - hosts:
    - mypc.com
    secretName: mypc-secret
  rules:
  - host: mypc.com

Accessories minion:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: mypc-accessories-minion
  namespace: accessories_ns
  annotations:
    nginx.org/mergeable-ingress-type: "minion"
spec:
  ingressClassName: nginx
  rules:
  - host: mypc.com
    http:
      paths:
      - path: /accessories
        pathType: Prefix
        backend:
          service:
            name: accessories-svc
            port:
              number: 80

Software minon:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: mypc-software-minion
  namespace: software_ns
  annotations:
    nginx.org/mergeable-ingress-type: "minion"
spec:
  ingressClassName: nginx
  rules:
  - host: mypc.com
    http:
      paths:
      - path: /software
        pathType: Prefix
        backend:
          service:
            name: software-svc
            port:
              number: 80

Hardware minion:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: mypc-hardware-minion
  namespace: hardware_ns
  annotations:
    nginx.org/mergeable-ingress-type: "minion"
spec:
  ingressClassName: nginx
  rules:
  - host: mypc.com
    http:
      paths:
      - path: /hardware
        pathType: Prefix
        backend:
          service:
            name: hardware-svc
            port:
              number: 80

For more Syntax and Rules regarding the Cross Namespace Ingress Configuration, refer here.

That’s it! Hope you have enjoyed the read and understood a bit more about Nginx Ingress in Kubernetes.