uptime 99
Published in

uptime 99

Making Sense of Kubernetes RBAC and IAM Roles on GKE

Google’s managed Kubernetes service has always been one of my favorite ways to run a Kubernetes cluster. Google Kubernetes Engine (GKE) provides some incredible functionality that makes Kubernetes even more enjoyable to work with. With that said, one of the best features of GKE is commonly misunderstood, the relationship between Google Cloud IAM and Kubernetes RBAC. When configured correctly, they can provide a simple and powerful approach to Kubernetes authorization.

RBAC Builds on IAM for GKE

With most services in Google Cloud, configuring IAM roles is all that’s needed to set up authorization. Although that’s also an option with GKE, you’ll likely need to add some RBAC configuration as part of an effective Kubernetes authorization strategy.

Essentially, all IAM configuration applies to your entire Google Cloud Project, and all clusters within that project. Kubernetes RBAC configuration applies to each cluster individually, and enables fine grained authorization at a namespace level. With GKE, these approaches to authorization work in parallel, with a user’s capabilities effectively representing a union of IAM and RBAC roles assigned to them.

Common IAM and RBAC Roles

Both Google Cloud IAM and Kubernetes RBAC provide default roles that act as a great starting point to any authorization strategy.

GCP IAM and Kubernetes RBAC have similarly named roles (cluster admin, admin, developer/edit, and viewer)

Based on the above graphic, you might think that the Google Cloud IAM roles on the left match up with the corresponding Kubernetes RBAC roles on the right. Although that’s mostly true, theres’ a significant variation when it comes to the cluster admin roles.

In Kubernetes RBAC, the cluster-admin role grants full administrative access to create, edit, or delete just about any resource in the cluster. With Google Cloud IAM, the Kubernetes Engine Cluster Admin role actually only grants the ability to change the cluster infrastructure, but not actually access any of the resources within a cluster. Of course both of these role names make sense in isolation, but given the overlap between the effects of IAM and RBAC, this naming can get confusing. Here’s an approximation of how the IAM roles actually relate to RBAC roles:

The GKE Admin role actually matches up more closely with the cluster-admin RBAC role while the GKE developer and viewer roles line up closely with RBAC edit and view roles.

These IAM roles don’t match the RBAC roles perfectly, but the capabilities in the roles these arrows connect generally line up pretty closely. It’s also worth noting that the project wide Owner, Editor, and Viewer IAM roles include the GKE Admin, Developer, and Viewer capabilities respectively (with a few interesting exceptions).

Using IAM and RBAC Together

With the Principle of Least Privilege as our guiding light, we can use a straightforward approach with both IAM and RBAC to achieve well rounded authorization. Since IAM roles grant access across all clusters in a GCP project, they should be used sparingly. As an example, granting the “Kubernetes Engine Developer” role will grant edit access to all namespaces, including kube-system, allowing core system components to be modified by anyone with this IAM role. Of course we still need to grant some form of basic access with IAM to enable a user to even get credentials to access a cluster. On GKE, RBAC is powerless without at least some IAM to grant that initial cluster access.

With this in mind, we generally recommend using the “Kubernetes Engine Viewer” IAM role to grant read only access. That access can can then be extended with granular RBAC configuration that grants write and/or admin access to specific namespaces. With GKE, IAM is the foundation for authorization and RBAC is the structure built on top of that.

Grant read only access with IAM, build on that with granular RBAC config.

In practice, that means granting edit or admin roles within your cluster to specific users at a namespace level to build on the read only access you’ve granted them with IAM. A simple role binding to grant Jane edit access within the web namespace would look something like this:

kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
name: jane-web
namespace: web
- kind: User
name: jane@example.com
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: edit
apiGroup: rbac.authorization.k8s.io

Defining RBAC configuration like this works relatively well at a small scale, but can quickly become difficult to manage across a variety of namespaces or users. It can also be difficult to get a full picture of authorization in your cluster when half of the configuration is in Google Cloud IAM while the other half lives in Kubernetes RBAC. After running into these problems consistently, we created some open source tools to help.

Simplifying RBAC Management

At ReactiveOps we manage clusters for a wide variety of organizations which means we’ve seen firsthand just how complex RBAC configuration can become at scale. To make matters worse, it can be quite difficult to automate changes to RBAC configuration in any meaningful way. Like many organizations, we try to use CI/CD and infrastructure as code wherever possible, that’s been really difficult with RBAC. Role bindings can’t be changed and instead need to be deleted and recreated when assigning a user a different role. Additionally, it becomes quite complex in any CI/CD workflow to notice the absence of a resource and make changes based on that. With RBAC, that’s a very important detail. Removing a role binding is required to revoke access, and that can get difficult to keep track of.

We built rbac-manager with all of this in mind. Built as a custom operator, rbac-manager can easily be deployed to your cluster and dramatically simplify your RBAC workflows. With rbac-manager’s custom resources (RBAC Definitions), we often see the amount of yaml cut to 1/3rd of the original configuration. Importantly, changing and revoking roles with rbac-manager all just works as you’d expect it to. If a role bound to a user is changed in a RBAC Definition, rbac-manager takes care of deleting and recreating the associated role binding with the new role. Additionally, if a role bound to a user is removed from a RBAC Definition, the associated role binding is automatically removed.

All of these improvements are relatively small, but when combined into a single tool, they can make RBAC much simpler to work with.

Simplifying RBAC (and IAM) Visibility

With the unique combination of RBAC and IAM that GKE offers, it can get difficult to get a full picture of who has access to a cluster. We created rbac-lookup to help with that. With a simple command like rbac-lookup rob, it will output all users, service accounts, or groups matching that name and show the RBAC roles they have been given in the cluster.

In our most recent release, we’ve introduced GKE IAM integration, that will also include the container IAM roles associated with these users in the output. So if you run rbac-lookup rob --gke it will include all the relevant RBAC roles, along with all the relevant IAM roles associated with GKE access for the matching users.

At ReactiveOps, we believe that it should be easy to configure and understand authorization on your Kubernetes clusters. We hope that both rbac-manager and rbac-lookup help make RBAC more approachable for everyone.




We collect, curate, and publish articles on everything cloud, kubernetes, open source, and security.

Recommended from Medium

How I learned Advanced Concepts in C while doing Projects

I would like to thank you for your airdrop.

An Intro to Git and GitHub for Beginners | Git for Newbies

Image result for git

Part 1: How to build a Job board with Vue js and the Laravel 5.5 API ? (Updated)

How much does it cost to build an app in 2022?

All the Good Practices I've learned in my Career

A Dockerized dbt Workflow

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Rob Scott

Rob Scott

Kubernetes, Docker, and more @ReactiveOps. Formerly @Spire.

More from Medium

GKE cluster scale zero — tips

Service to Service call pattern — Using Anthos Service Mesh

POC: Endpoint Monitoring using Prometheus and Grafana

Deploying Calico with eBPF on OpenShift — CNCF Roadmap