Kubernetes: user authorization with certificates

Published in

Krateo PlatformOps

5 min readMar 6

In Krateo (krateo.io) we are thinking how to implement the part of user authorization.

We have spent a lot of time evaluating if we need to create custom authorization or to use the native kubernetes rbac, and we have decided to proceed with the latter.

The idea

In this example, we want to create 3 different groups:

dev -> can read pods
ops -> can read configmaps
admin -> can read secrets

But only two groups we will bind to Kim, so this is our goal:

user (kim) -> group (dev) -> ClusterRoleBinding -> ClusterRole -> pods
user (kim) -> group (ops) -> ClusterRoleBinding -> ClusterRole -> configmaps

Remember: Kim will have grants to access to pods and to configmaps, but NOT to secrets.

Generate the Kim’s certificate

The first step is to generate a certificate, and to do this we will use the openssl client:

> openssl genrsa -out kim.key 2048

With this new certificate, we generate a .csr file that contains info about Kim-group binding. We want to bind Kim to dev group and ops group, but NOT to admin group.

> openssl req -new -key kim.key -out kim.csr -subj "/CN=kim/O=dev/O=ops"

After this, we need to encode in base64 the certificate content:

> cat kim.csr | base64

This is the output of the previous command:

LS0tLS1CRUdJTiBDRVJUSUZJQ0FURSBSRVFVRVNULS0tLS0KTUlJQ2J6Q0NBVmNDQVFBd0tqRU1NQW9HQTFVRUF3d0RhMmx0TVF3d0NnWURWUVFLREFOa1pYWXhEREFLQmdOVgpCQW9NQTI5d2N6Q0NBU0l3RFFZSktvWklodmNOQVFFQkJRQURnZ0VQQURDQ0FRb0NnZ0VCQUxzellJY3dYRDBFCnlmWkxlODhDMElCRnFrU3VBVXlienh4UjY3cEdZNnhmTVdxaEh5S0hOQXlxTWs2NjZuekszZmxXZjB2Rzc3cHYKbUY5WTN3U1NpTjBFa1NvRHM4MFFPanpmNENiWlJIejY1NDVodUZZVG9hOE5zUUZGcG85M0FWOHQwMUFOUUNXQgpYUjRvRTlsNmEyRnNwRFNhSGErQ2JQU1dHN3FRdDh4TnkrK3NuSFJEUWJ4UWZsallmeGNoUjE5MzJrWmkxbUVlClNidzNFN3M5cHlNTU1oWlNCTWVsWW0zT1I4M2didU02LzBZQm5Fa3lyTUIwa0pPcHVIbVprdkVGMkExbmhQUW0KbnA3MnZiU1pndGdmVGNHazZJRzBmWjBLZnZFM2F6ZEpyTUY2dTZBUHE1QnhTdGFVc2dBMDB1d2pySXYwYXcrKwpHQzlUdk1RZnVtRUNBd0VBQWFBQU1BMEdDU3FHU0liM0RRRUJDd1VBQTRJQkFRQWFvVFJhMEw4WFFqMUlGWG9jCk53RlQzOHZKdW8wN0JiMzNHMWtqWWFubTRJazJ6eDRDdlpXdytDQzFueEZ6OWxrdDEwNjhzRzB3Z3hKK2Jaa3EKNEZYUTFBK294c1V5NG9vRXlNcFBDMHQ4STUwYzg0a2Q3WHR3c0RGK2d3dmFaRjcxYmp3Zi9ReURUR2pUWExScQpueHJ1QUIyN0ZXd1k2WnlBOUlEeVovZEI0SHRtNzcySGNuRkVvRXZFYjZ1UGZZR2JHTVRoS1pTZGxYcDRraXlPCmVqNmhsNElJYm00YmJjS1FPeHFFV1NENjVENjZwaVBKS1VHLzBvN0FtQ2RKYUxhMHpBZFFxSnFjb2liWjZPV1oKV2JIeEJNY1lzcU5qVlN5MWZJTlN5cWRtd3M3WkhWaTNycnhoZTk3dHZGWnNjYVRUR1kyNm5HV0k0VFduMlFkRQpCaDVYCi0tLS0tRU5EIENFUlRJRklDQVRFIFJFUVVFU1QtLS0tLQo=

The Certificate Signing Request

Let’s use the generated certificate and create the request.yaml file.

apiVersion: certificates.k8s.io/v1
kind: CertificateSigningRequest
metadata:
  name: kim
spec:
  request: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURSBSRVFVRVNULS0tLS0KTUlJQ2J6Q0NBVmNDQVFBd0tqRU1NQW9HQTFVRUF3d0RhMmx0TVF3d0NnWURWUVFLREFOa1pYWXhEREFLQmdOVgpCQW9NQTI5d2N6Q0NBU0l3RFFZSktvWklodmNOQVFFQkJRQURnZ0VQQURDQ0FRb0NnZ0VCQUxzellJY3dYRDBFCnlmWkxlODhDMElCRnFrU3VBVXlienh4UjY3cEdZNnhmTVdxaEh5S0hOQXlxTWs2NjZuekszZmxXZjB2Rzc3cHYKbUY5WTN3U1NpTjBFa1NvRHM4MFFPanpmNENiWlJIejY1NDVodUZZVG9hOE5zUUZGcG85M0FWOHQwMUFOUUNXQgpYUjRvRTlsNmEyRnNwRFNhSGErQ2JQU1dHN3FRdDh4TnkrK3NuSFJEUWJ4UWZsallmeGNoUjE5MzJrWmkxbUVlClNidzNFN3M5cHlNTU1oWlNCTWVsWW0zT1I4M2didU02LzBZQm5Fa3lyTUIwa0pPcHVIbVprdkVGMkExbmhQUW0KbnA3MnZiU1pndGdmVGNHazZJRzBmWjBLZnZFM2F6ZEpyTUY2dTZBUHE1QnhTdGFVc2dBMDB1d2pySXYwYXcrKwpHQzlUdk1RZnVtRUNBd0VBQWFBQU1BMEdDU3FHU0liM0RRRUJDd1VBQTRJQkFRQWFvVFJhMEw4WFFqMUlGWG9jCk53RlQzOHZKdW8wN0JiMzNHMWtqWWFubTRJazJ6eDRDdlpXdytDQzFueEZ6OWxrdDEwNjhzRzB3Z3hKK2Jaa3EKNEZYUTFBK294c1V5NG9vRXlNcFBDMHQ4STUwYzg0a2Q3WHR3c0RGK2d3dmFaRjcxYmp3Zi9ReURUR2pUWExScQpueHJ1QUIyN0ZXd1k2WnlBOUlEeVovZEI0SHRtNzcySGNuRkVvRXZFYjZ1UGZZR2JHTVRoS1pTZGxYcDRraXlPCmVqNmhsNElJYm00YmJjS1FPeHFFV1NENjVENjZwaVBKS1VHLzBvN0FtQ2RKYUxhMHpBZFFxSnFjb2liWjZPV1oKV2JIeEJNY1lzcU5qVlN5MWZJTlN5cWRtd3M3WkhWaTNycnhoZTk3dHZGWnNjYVRUR1kyNm5HV0k0VFduMlFkRQpCaDVYCi0tLS0tRU5EIENFUlRJRklDQVRFIFJFUVVFU1QtLS0tLQo=
  signerName: kubernetes.io/kube-apiserver-client
  expirationSeconds: 864000 # one day
  usages:
    - client auth

As you can see the “request” property contains the base64 of the kim.csr content, more info about this yaml can be found here https://kubernetes.io/docs/reference/access-authn-authz/certificate-signing-requests/#authorization.

Note for AWS users:
If you are running kubernetes on EKS, you need to change the .spec.signerName to “beta.eks.amazonaws.com/app-serving”

> kubectl apply -f request.yaml
certificatesigningrequest.certificates.k8s.io/kim created

Get current CSR (CertificateSigningRequest):

> kubectl get csr
NAME   AGE   SIGNERNAME                            REQUESTOR            REQUESTEDDURATION   CONDITION
kim    8s    kubernetes.io/kube-apiserver-client   docker-for-desktop   10d                 Pending

As you can see, the request is in “pending” condition, so let’s approve it.

> kubectl certificate approve kim
certificatesigningrequest.certificates.k8s.io/kim approved

> kubectl get csr
NAME   AGE    SIGNERNAME                            REQUESTOR            REQUESTEDDURATION   CONDITION
kim    101s   kubernetes.io/kube-apiserver-client   docker-for-desktop   10d                 Approved,Issued

Well done, so now, we need to get the certificate generated by our approve and save it to a new file named “kim.crt”.

> kubectl get csr kim -o jsonpath='{.status.certificate}' | base64 --decode > kim.crt

And this is the content of the kim.crt file

We also want to create a secret with the content of the file kim.key (why will be explained later in the programmatically section)

> kubectl create secret generic kim  --from-file=key=kim.key

ClusterRole and ClusterRoleBinding

As previously mentioned, we need to create 3 cluster role (group) and 3 cluster role binding with relative permissions.

ClusterRole.yaml

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: secret-reader
rules:
  - apiGroups: [""]
    resources: ["secrets"]
    verbs: ["get", "watch", "list"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: configmap-reader
rules:
  - apiGroups: [""]
    resources: ["configmaps"]
    verbs: ["get", "watch", "list"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: pod-reader
rules:
  - apiGroups: [""]
    resources: ["pods"]
    verbs: ["get", "watch", "list"]

ClusterRoleBinding.yaml

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: read-secrets-global
subjects:
  - kind: Group
    name: admin # This is the group name
    apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: ClusterRole
  name: secret-reader
  apiGroup: rbac.authorization.k8s.io
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: read-configmaps-global
subjects:
  - kind: Group
    name: ops # This is the group name
    apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: ClusterRole
  name: configmap-reader
  apiGroup: rbac.authorization.k8s.io
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: read-pods-global
subjects:
  - kind: Group
    name: dev # This is the group name
    apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: ClusterRole
  name: pod-reader
  apiGroup: rbac.authorization.k8s.io

Apply both to your cluster:

> kubectl apply -f ClusterRole.yaml
> kubectl apply -f ClusterRoleBinding.yaml

Create and use context

We have all we need to login in the cluster as Kim, now create the context for your .kube file:

> kubectl config set-credentials kim --client-key=kim.key --client-certificate=kim.crt --embed-certs=true
User "kim" set.

> kubectl config set-context kim --cluster=docker-desktop --user=kim
Context "kim" created.> kubectl config use-context kim
Switched to context "kim".

Note: in the second command we have defined the “ — cluster” flag, in this example we are working on docker desktop, change it to your cluster name.

Now, we can test our work and try to get pods, configmaps and secrets as kim user:

> kubectl get po                                                                                                                                                                                    1 

NAME      READY   STATUS      RESTARTS   AGE
busybox   0/3     Completed   0          3d3h

> kubectl get cm 
NAME               DATA   AGE
kube-root-ca.crt   1      4d17h

> kubectl get secret
Error from server (Forbidden): secrets is forbidden: User "kim" cannot list resource "secrets" in API group "" in the namespace "default"

Use certificate programmatically (NodeJS sample)

Now we want to use the kim’s certificate inside a NodeJS program, in this example we will use the official javascript kubernetes client (https://github.com/kubernetes-client/javascript).

To authenticate with kim’s certificate we need to take the certificate in the CertificateSigningRequest and the content of the kim.key that we used to request it, so is this the reason why we have created the secret in the previous steps.

The program is very easy, but it is used to demonstrate how to use the certificate previously generated.

We have splitted the code in two functions:

main(user)
getConfigMaps(cert, key)

The index.js file is something like this:

const request = require('request')
const k8s = require('@kubernetes/client-node')

const main = async (user) => {
  // ...
}const getConfigMaps = async (cert, key) => {
  // ...
}main('kim')

This is the main function:

const main = async (user) => {
  const kc = new k8s.KubeConfig()
  kc.loadFromDefault()
  const opts = {}
  kc.applyToRequest(opts)

  const apis = [
    `/apis/certificates.k8s.io/v1/certificatesigningrequests/${user}`,
    `/api/v1/namespaces/default/secrets/${user}`
  ]  const result = await Promise.all(
    apis.map(async (path) => {
      const url = encodeURI(`${kc.getCurrentCluster().server}${path}`)
      return await new Promise((resolve, reject) => {
        request(url, opts, (error, response, data) => {
          if (error || response.statusCode !== 200) {
            reject(error)
          } else {
            resolve(JSON.parse(data))
          }
        })
      })
    })
  )  const cert = result[0].status.certificate
  const key = result[1].data.key
  getConfigMaps(cert, key)
}

This is the getConfigMaps function:

const getConfigMaps = async (cert, key) => {
  const kc = new k8s.KubeConfig()
  kc.loadFromClusterAndUser(
    {
      name: 'docker-desktop',
      server: 'https://kubernetes.docker.internal:6443',
      skipTLSVerify: true
    },
    {
      certData: cert,
      keyData: key
    }
  )

  const opts = {}
  kc.applyToRequest(opts)  const k8sApi = kc.makeApiClient(k8s.CoreV1Api)  // use .listSecretForAllNamespaces() to test secrets access
  k8sApi
    .listConfigMapForAllNamespaces()
    .then((res) => {
      console.log('### ConfigMaps ###')
      res.body.items.forEach((item) => {
        console.log(item.metadata.name)
      })
    })
    .catch((err) => {
      console.log(err.body.message)
    })
}

And this is our output:

> node index.js

### ConfigMaps ###
kube-root-ca.crt
kube-root-ca.crt
cluster-info
kube-root-ca.crt
coredns
extension-apiserver-authentication
kube-proxy
kube-root-ca.crt
kubeadm-config
kubelet-config

Please note that if you want to use this code inside a pod in the cluster, you must correctly assign permission to the pod to get CSR and secrets, and the server url (not covered in this article).

Cover mage courtesy by vectorjuice on Freepik.

Kubernetes: user authorization with certificates

The idea

Generate the Kim’s certificate

The Certificate Signing Request

ClusterRole and ClusterRoleBinding

Create and use context

Use certificate programmatically (NodeJS sample)

Written by Mauro Sala

More from Mauro Sala and Krateo PlatformOps

Krateo recognised as a Representative Vendor in the 2023 Gartner® Market Guide for Internal…

VERONA, Italy — September 26, 2023 — Krateo, has been named a Representative Vendor in the Gartner Market guide for IDPs.

Platform Engineering requires standardization over consolidation — that’s why we wrote Krateo…

It was back in early 2021 the exact moment when we realized we were living a pivotal moment in the cloud native digital transformation…

Introducing Krateo: Revolutionizing the Way Businesses Manage and Optimize their Platforms

[Verona, June 6th] — We are thrilled to announce Krateo S.r.l. and the launch of Krateo PlatformOps, our software to optimise and make…

Recommended from Medium

Grafana Loki — Our journey on replacing Elastic Search and adopting a new logging solution at…

Logging is an important aspect of any software system. As a company grows, collecting, storing, and presenting logs can become a challenge…

“Containerization Using Docker”

If you’re new to Docker or simply looking to refresh your knowledge, this blog post serves as a comprehensive introduction to Docker…

Lists

General Coding Knowledge

New_Reading_List

Productivity

Natural Language Processing

Using NGINX as API Gateway

Hi everyone!

Demystifying Kubernetes RBAC: A Deep Dive into Role-Based Access Control

Introduction

OPA Gatekeeper Use-Cases: Ensure Pull Only From Specific Allowed Container Registries in your…

In this article we will see how to ensure that only images from specific container registries are allowed into your kubernetes cluster!

Kubernetes “x509: certificate has expired or is not yet valid” error

This is going to be a short article and I want to write it because of the pain that it took me to fix this error.