Using Workload Identity to access Google Cloud Pub/Sub from Google Kubernetes Engine

In this article we will see how to publish a message to a Pub/Sub topic from a Node.js application running on Google Cloud Kubernetes Engine. We will use Workload Identity to access Pub/Sub service from the node application.

Why Workload Identity ?

https://cloud.google.com/kubernetes-engine/docs/concepts/workload-identity

What is Workload Identity ?

https://cloud.google.com/kubernetes-engine/docs/concepts/workload-identity

Google Cloud Services that we will be using

• Pub/Sub
• Kubernetes Engine
• IAM

The node app has an endpoint (/publish) to publish a message to the Pub/Sub topic, the code is available here.

IAM

Create a service account in IAM, and assign the Pub/Sub Publisher permissions to it.

Pub/Sub

Create a topic with default subscription

Kubernetes Engine

Create a GKE cluster using the Autopilot mode. For Autopilot, the Workload Identity is enabled by default.

Wait for the cluster to be ready.

Once the cluster is ready, connect to it. We can get the command to connect from the console,

connect and create a Kubernetes service account(KSA)

kubectl create serviceaccount pub-sub-publisher-k8s

We have to create a binding between Kubernetes service account and the IAM service account, this binding will allow the Kubernetes service account to act as IAM service account

gcloud iam service-accounts add-iam-policy-binding pub-sub-publisher@rational-terra-336303.iam.gserviceaccount.com \
    --role roles/iam.workloadIdentityUser \
    --member "serviceAccount:rational-terra-336303.svc.id.goog[default/pub-sub-publisher-k8s]"

Command format:

https://cloud.google.com/kubernetes-engine/docs/how-to/workload-identity

Next, we have to annotate the Kubernetes service account with email address of the IAM service account

kubectl annotate serviceaccount pub-sub-publisher-k8s \
    --namespace default \
    iam.gke.io/gcp-service-account=pub-sub-publisher@rational-terra-336303.iam.gserviceaccount.com

Command format:

https://cloud.google.com/kubernetes-engine/docs/how-to/workload-identity

Now, we can use the K8s service account in the pod specification ( deployment.yaml ). This would allow the K8s service account to act as the IAM service account (we assigned the Pub/Sub Publisher permissions to it) and our node app can publish messages to the Pub/Sub topic.

The container image mentioned in the pod specification is the docker image of node app and this image is stored in the Artifact Registry, If you are new to Artifact registry please refer to one of my articles to know more about it.

Testing

Create the Kubernetes objects defined in the deployment.yaml file

kubectl apply -f deployment.yaml

Once the service is ready, access the URL with publish endpoint to publish a message to the Pub/Sub topic

curl http://34.87.245.6:5050/publish
Result:
Message 3902383287545125 published

Navigate to the Pub/Sub service and go to the default subscription that was created with our topic. Click Pull to view the messages,

we should see the message that was sent by the node app

Conclusion

That’s it ! we have seen how easy it is to use the Workload Identity to access Google Cloud service (Pub/Sub) from GKE workload.