Kubernetes — Authenticating to your cluster using Keycloak

Accessing a Kubernetes cluster and its resources is done via API calls to the Kubernetes API. Whenever we make a call to the API server, it first needs to be authenticated. Once it is authenticated, it then needs to be authorized. The authentication part ensures the system knows about the user, the authorization checks and validates whether the user has the correct permissions for the call being made.

At ELMO Software, we increasingly found development teams moving onto the new Kubernetes infrastructure. This meant we needed to find an easy and secure way to control different levels of access to the clusters. We already had Keycloak implemented for various internal tools, so it made sense to continue to use Keycloak for Kubernetes authentication. We can use Keycloak to pass group claims onto Kubernetes and therefore use different RBAC rules to determine different permissions. For example, we allow the infrastructure team to have full cluster-admin access but the development teams only get full access to specific namespaces. The guide below will show you how to set all of this up.

This guide assumes that you already have a running Kubernetes cluster and that you already have a Keycloak server setup.

Configuring our Kubernetes API server

We need to configure the OIDC settings in our Kubernetes API server. This can be done by changing the apiserver flags. If you are using Kops you can use the below cluster config:

spec:
  kubeAPIServer:
    oidcClientID: kubernetes
    oidcGroupsClaim: groups
    oidcGroupsPrefix: 'keycloak:'
    oidcIssuerURL: https://<keycloakserverurl>/auth/realms/master
    oidcUsernameClaim: email

Make sure in the above config your realm is set correctly.

If you are not using kops you can use the flags in this document and pass them to the API server manually.

Setting up Keycloak

1. First, we want to set up a client in Keycloak, so go to Clients on the navbar and then create a new client:

2. Copy the below settings into the client settings:

• Access type — Change to confidential
• Valid redirect URIs — Add HTTP://localhost:18000 and HTTP://localhost:8000

3. We next want to create some groups/roles for more fine-grained access to our cluster. We can then use these roles to map RBAC rules. Go to the Roles tab of the client we just created and then create a new role. We will create an admin role and a developer role:

• Admin — This role will be given to people who need cluster admin access.
• Developer — This role will be given to developers and have limited access compared to the admin role.

Feel free to create whatever roles you need for your cluster.

4. Next, head over to the Mappers tab and create a new Mapper with the following settings:

• This allows us to pass the role names into the request to the Kubernetes API server.

5. Now the fun part of assigning the roles to our users. Go to the Users tab and find the user you wish to assign a role to. You will need to go to the Role Mappings tab and then choose Kubernetes in the client roles:

• I have given my clusteradmin user the admin role.

That is is for Keycloak setup for now. We will be back with it in one of the below sections.

Setting up RBAC rules

If you are unsure of what RBAC in Kubernetes is, have a read here before continuing.

First, we want to create a cluster role binding to bind our keycloak:admin role to our cluster:admin role.

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: admin-role-binding
subjects:
- kind: Group
name: keycloak:admin
apiGroup: rbac.authorization.k8s.io
roleRef:
kind: ClusterRole
name: cluster-admin
apiGroup: rbac.authorization.k8s.io

• This is taking our keycloak:admin group that we created in keycloak and assigned to our user and then binding it to the cluster-admin role provided by Kubernetes.

We will now create a developer role that has restricted access to a particular namespace:

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
annotations:
rbac.authorization.kubernetes.io/autoupdate: "true"
labels:
name: ns-read-only
namespace: kube-system
rules:
- apiGroups:
- ""
resources: ["namespaces"]
verbs:
- list

• We create a clusterrole that only gives list permissions to namespaces.

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: developer-readonly-ns
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: ns-read-only
subjects:
- apiGroup: rbac.authorization.k8s.io
kind: Group
name: keycloak:developer
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: developer-role
namespace: test-ns
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: cluster-admin
subjects:
- apiGroup: rbac.authorization.k8s.io
kind: Group
name: keycloak:developer

• Here we are giving our developer role a clusterrolebinding to the namespace readonly role.
• We are then giving just a Role Binding the developer role in the namespace test-ns.
• Basically, we are giving our developer full access to only the test-ns namespace.

You can configure your RBAC however you wish, as long as your binding is to the Group kind and is for your role.

Connecting to the cluster

To connect to the cluster using OIDC we use kubelogin:

int128/kubelogin

Install kubelogin before continuing:

1. Go to keycloak again and then go back to the Kubernetes client we created. You will see a tab called Credentials, go here and grab the client secret.
2. Run the below command in your terminal to verify authentication to keycloak:

kubectl oidc-login setup \
--oidc-issuer-url=https://<keycloak_server-url>/auth/realms/demo \
--oidc-client-id=kubernetes \
--oidc-client-secret=<CLIENT SECRET FROM ABOVE>

3. If everything is set up correctly your browser will be opened asking you to log in and you will get an authenticated screen once done:

In your terminal you should see the token with all the claims:

{
  "exp": 1625191970,
  "iat": 1625191670,
  "auth_time": 1625190917,
  "jti": "6dfba74a-21a0-4b8b-aa28-1a1760f33108",
  "iss": "https://<KEYCLOAK_URL>/auth/realms/Demo",
  "aud": "kubernetes",
  "sub": "4a042ec2-db0c-4093-a448-cc6a60362c13",
  "typ": "ID",
  "azp": "kubernetes",
  "nonce": "9pLW1vEC3ZLrORTHH69QgQU3BvGTzeP2Rynr6obNK3s",
  "session_state": "c115727c-8c15-45a3-9221-622ddb06e714",
  "at_hash": "hUtGwB_qIqCwZk1HbALNTg",
  "acr": "0",
  "email_verified": true,
  "groups": [
    "admin"
  ],
  "preferred_username": "clusteradmin",
  "email": "clusteradmin@demo.com"
}

• We can see our groups claim here and our email. We are all good to go on setting up our kubeconfig file.

4. Go to your kubeconfig file and add a user like below:

- name: keycloak
  user:
    exec:
      apiVersion: client.authentication.k8s.io/v1beta1
      args:
      - oidc-login
      - get-token
      - --oidc-issuer-url=https://<KEYCLOAK_SERVER>/auth/realms/master
      - --oidc-client-id=kubernetes
      - --oidc-client-secret=<CLIENT_SECRET>
      command: kubectl
      env: null
      provideClusterInfo: false

• Change your cluster to use this keycloak user

- context:
    cluster: example-cluster
    user: keycloak
  name: example-cluster

5. Now run some kubectl commands and off you go!

My next blog will take you through setting up Kubernetes Audit Logging using the Elastic Stack and how you can use this to monitor user activity.