Using Cloud NAT with GKE Cluster

Days of building NAT compute engine instance with routes for connecting private GKE cluster to internet has gone. But my previous post on using NAT is still relevant if we assign public ip to cluster nodes.

With the introduction of Cloud NAT(Beta release), compute engine instances and GKE cluster nodes without public ip i.e. private instances can connect to internet through cloud NAT using static or dynamic ip addresses. NAT gateway is regional resource which can be configured to be used for all or selected subnets in the region.

image: ciobulletin.com

In this post, we will walk through creating private cluster, creating NAT gateway, testing connection and confirming the gateway IP.

Create Private GKE Cluster

We are going to create cluster named private-cluster at us-central1-b zone. The master of private cluster should be supplied with authorized network otherwise we can’t connect to the api server i.e. cannot run kubectl command.

Let’s use cloud shell ip as authorized network. Get the IP address of the cloudshell with curl checkip.amazonaws.com and keep it on [cloudshell-ip-address] variable below. Also, adjust other parameters as required.

gcloud container clusters create private-cluster \
    --create-subnetwork name=private-cluster \
    --enable-ip-alias \
    --enable-private-nodes \
    --master-ipv4-cidr 172.16.0.0/28 \
    --enable-master-authorized-networks \
    --master-authorized-networks [cloudshell-ip-address]
    --no-enable-basic-auth \
    --no-issue-client-certificate

It takes few minutes to get the cluster up and running. Head over to kubernetes list page, click on connect for the newly created cluster and then Run in Cloud Shell. This should open cloud shell with following command:

gcloud container clusters get-credentials private-cluster --zone us-central1-b

Try Connecting to Internet Without NAT

Create a simple deployment of curl for getting the ip address that is used to connect public internet.

kubectl run test-deploy --image dwdraju/alpine-curl-jq

Checkout if the pod is created

kubectl get pods
NAME                           READY     STATUS         RESTARTS   AGEtest-deploy-7857c9c7c9-ffpsl   0/1       ErrImagePull   0          18s

That means we cannot pull any image to the cluster as none of the nodes have external ip to connect to docker registry. But it can pull image of Cloud Registry if have access.

Create Cloud NAT Gateway

Cloud NAT is one of Network Services offered by Google Cloud Platform(GCP). Go to the NAT creation page and start adding details.

If you don’t have cloud router, add one from the NAT addition page along with subnet, region. For NAT Mapping, select Custom and choose the subnet where we created our private cluster, Selected: all or choose IP ranges.

For the NAT IP address, choose automatic for giving freedom to add ip address to the GCP platform or create a new static ip in the same region to use pre-defined ip address. Hit Create and wait for sometime to get the NAT gateway ready.

Create new deployment to check the cluster can pull image from docker hub registry.

kubectl run nat-deploy --image dwdraju/alpine-curl-jq
kubectl get pods

The pod status will be seen marked completed.

Get the IP address of NAT, Confirm its Working

Run a pod get-ip-address as interactive shell

kubectl run -i --tty get-ip-address --image=dwdraju/alpine-curl-jq --restart=Never

On the interactive terminal, enter curl checkip.amazonaws.com which will output the ip address of the NAT gateway we created.

kubectl run -i --tty get-ip-address --image=dwdraju/alpine-curl-jq --restart=Never
If you don't see a command prompt, try pressing enter.
/ # curl checkip.amazonaws.com
104.198.47.200

That’s all !! By this way we have our secure cluster which cannot be intercepted by outside world but our nodes can get public internet resources using the NAT IP address. This is also helpful if you have to whitelist ip address for the resource where kubernetes resource(pod) want to connect to.

To make billing don’t scare you, delete the cluster and NAT gateway :)