Using nginx-ingress controller to restrict access by IP (ip whitelisting) for a service deployed to a Kubernetes (AKS) cluster

While working on a project earlier this week we were given the following requirements :

  1. Create a Managed Kubernetes cluster (AKS) on Azure, in a existing Azure Subnet using an ARM template / az commands
  2. For a service deployed to the AKS cluster, restrict access to certain client source IPs (Ip Whitelisting)
  3. For a service deployed to the AKS cluster auto provision Letsencrypt TLS certificates. You can follow the post for this

This post details point 2 above. I will be creating a separate post for point 1 and 3 above, in the near future.

The rest of this post assumes that the AKS Kubernetes cluster is available, you have helm installed, and we have already executed the helm init command.


Create or update the nginx-ingress controller

The first thing we do now is install the inginx-ingress controller using helm. The github page for the nginx-ingress controller helm chart is at nginx-ingress. The install command to be used is :

Ingress controller install command

$ helm install stable/nginx-ingress  --set controller.service.externalTrafficPolicy=Local

The default value of controller.service.externalTrafficPolicy in the nginx ingress helm chart is ‘Cluster’, we need to change this value to ‘Local’. With the default value of ‘Cluster’ the ingress controller does not see the actual source ip from the client request but an internal IP. After setting this value to ‘Local’ the ingress controller gets the unmodified source ip of the client request.

Before we apply the ingress rule with source ip whitelisting for a service, let us create a sample web app deployment and service:


Create the the hello world web server deployment and service to test the whitelisting

Create Deployment

$ kubectl run web --image=tutum/hello-world --port=80 

Expose the deployment as a service

$ kubectl expose deployment web --port=80

Check if deployment service and pods are up

$ kubectl get deployment webNAME      DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGEweb       1         1         1            1           17m$ kubectl get svc webNAME      CLUSTER-IP    EXTERNAL-IP   PORT(S)   AGEweb       10.3.30.224   <none>        80/TCP    5m$ kubectl get pod -l=run=webNAME                   READY     STATUS    RESTARTS   AGEweb-5bff8ffd8c-twxwp   1/1       Running   0          17m

Apply the ingress IP Whitelisting rule for the service

The annotation ( nginx.ingress.kubernetes.io/whitelist-source-range )we need to apply to the kubernetes ingress resource using nginx-ingress is detailed at nginx-ingress.

We now create the ingress rule

Ingress YAML file

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: web-ingress
annotations:
ingress.kubernetes.io/whitelist-source-range: 49.36.X.X/32
spec:
rules:
- host: web.manitestdomain.com
http:
paths:
- path: /
backend:
serviceName: web
servicePort: 80

Apply the ingress rule

$ kubectl apply -f web-ingress-whitelist.yaml

Thats it whitelisting is in place.


Testing and Debugging the whitelisting rules

Before steps below were executed dns configurations were modified so that the domain manitestdomain.com was pointing to the public ip of the nginx-ingress controller service.

Request from whitelisted ip

$ curl ipinfo.io/ip
49.36.X.X
$ curl -I web.manitestdomain.com
HTTP/1.1 200 OK
Server: nginx/1.13.5
Date: Sat, 19 May 2018 06:07:01 GMT
Content-Type: text/html; charset=UTF-8
Connection: keep-alive
X-Powered-By: PHP/5.6.14

Request from Non Whitelisted ip

$ curl ipinfo.io/ip223.Y.Y.Y$ curl -I web.manitestdomain.com
HTTP/1.1 403 Forbidden
Server: nginx/1.13.5
Date: Sat, 19 May 2018 06:15:47 GMT
Content-Type: text/html
Content-Length: 169
Connection: keep-alive

Checking nginx-ingress controller logs

$ kubectl get pods
NAME READY STATUS RESTARTS AGE
nginx-ingress-nginx-ingress-controller-586c47b885-rxm72 1/1 Running 0 1d
nginx-ingress-nginx-ingress-default-backend-65f4cd97fb-sbh7c 1/1 Running 0 1d
web-5bff8ffd8c-twxwp
$ kubectl logs nginx-ingress-nginx-ingress-controller-586c47b885-rxm72 -f..
..
..
49.36.X.X - [49.36.X.X] - - [19/May/2018:06:11:21 +0000] "HEAD / HTTP/1.1" 200 0 "-" "curl/7.52.1" 87 0.002 [default-web-80] 10.2.1.7:80 0 0.002 200

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch

Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore

Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store