Kubernetes HTTP API with Authentication and Authorization
happy devSecOps
Kubernets API Server
The core of Kubernetes’ control plane is the API server. The API Server exposes a HTTP API that lets end users, different parts of your cluster, and external components communicate with one another. It lets you query and manipulate the state of API objects in Kubernetes (for example: Pods, Namespaces, ConfigMaps, and Events). The API Server is the main management point of the entire cluster. In short, it processes REST operations, validates them, and updates the corresponding objects in etcd
. The API Server serves up the Kubernetes API and is intended to be a relatively simple server, with most business logic implemented in separate components or in plugins. Following are some example APIs exposed in API Server.
When you interact with your Kubernetes cluster using the kubectl
command-line interface, you are actually communicating with the master API Server component. kubectl
commands converted to HTTP REST calls and invoke in the API Server. The API Server is also responsible for the authentication and authorization mechanism. All API clients should be authenticated in order to interact with the API Server. You can write Kubernetes client libraries/applications using this API server(e.g your own kubectl
, your own chaos engineering system
for k8s). Following figures shows the architecture and request handling flow of API Server.
In this post I’m goanna discuss about accessing Kubernets API functions via curl
command-line utility. All the deployments which related to this post available in gitlab. Please clone the repo and continue the post.
API Server Host and Port
To call any API, you need to know its server address first. In the case of Kubernetes, there is an API server per cluster(I have deployed the K8s cluster with Minikube
). Thus, the easiest way to find the API host and port is to look at the kubectl cluster-info
output. As shown in following example the API Server running on https://192.168.64.30:8443
. 192.168.64.30
is the Minikube IP, it expose HTTPS API on port 8443
.
API Server Authentication/Authorization
By default, Kubernetes exposes its API via HTTPS
, in particular, to guarantee a strong identity of the API Server to the clients. To invoke this API, the client requests needs implement the correct auth(authentication and authorization). There are two main authentication mechanisms we can use handle the API Server auth, 1) SSL Certificate-based auth
, 2) Token-based auth
. I have demonstrated the accessing Kubernetes API Server functions(APIs) with these two auth methods using curl
command-line utility.
SSL-Certificate-based Auth
The first way to authenticate API requests is via SSL certificates. As I have mentioned, I have deployed the Kubernetes cluster with Minikube. The Minikube bootstrapped the local cluster using a self-signed certificate
. Thus, the TLS cert of the Kubernetes API server turned out to be signed by a Certificate Authority (CA) MinikubeCA
. The certificates signed by this MinikubeCA
is trusted by the Kubernetes API Server. However this MinikubeCA
is unknown to curl
. By default, curl
trusts the same set of CAs the underlying operating system does. For instance, on Ubuntu or Debian, the list of trusted CAs can be found at /etc/ssl/certs/ca-certificates.crt
. Minikube doesn't add its certs to this file. So first we need to add the MinikubeCA certificate which locates in the ~/.minikube/ca.crt
to the curl
command.
Next we need to define the client certificate information in the curl
request. When bootstrapped the cluster, Minikube also created a user. This user got a certificate signed by the same MinikubeCA authority. Since this CA is trusted by the Kubernetes API server, presenting this certificate in the request will make it authenticated as the said user. Basically, any user that presents a valid certificate signed by the cluster’s Certificate Authority is considered authenticated. The client certificate locates inside the ~/.minikube/profiles/minikube/client.crt
. The client private key locates inside the ~/.minikube/profiles/minikube/client.key
. We need to set client certificate and client key in the curl
request to authenticate with API Server.
Token-based Auth
Another way to authenticate API requests is through using a bearer
header containing a valid Service Account JWT token. Much like with users, different Service Accounts will have different levels of access. I have discussed detailed information about the Service Accounts and access level(Roles
, ClusterRoles
etc) in my previous blog post. I’m not gonna discuss details about Kubernetes Service Account and permission handling in this post since I have discussed detailed information in my previous blog. Please refer that to get more details. The Service Account has a token. This token is stored as a Kubernetes Secret and can be read as a Secret. This token is what you’ll use to authenticate your third-party app to the Kubernetes API Server. In this scenario I have created a Kubernetes Service Account named bassa-serviceaccount
. Then I have defined cluster wide permissions(e.g to get, list, watch
the pods, services, namespaces, deployments, jobs
) with CluserRole
. Finally this ClusterRole
assigned to the Service Account via ClusterRoleBinding
. Now the token inside Service Account can be used to authenticate/authorization the API Server. I can invoke Kubernetes API Server APIs with this token(with bearer
header) and access the resources(e.g pods, services, namespaces
etc).
Following is the way to create Service Account. I have assigned this Service Account into a namespace called bassa
. Then viewed the Service Account token which can be used as JWT token. Even though I have assigned the Service Account to a namespace, the token inside the Service Account can be used to access the cluster wide resources via the API Service since I have assigned cluster-wide permissions with ClusterRole.
Following is the way to create ClusterRole
with different permissions. This Role defines the actions that can be performed(get
, watch
, list
) for the resources pods
. The namespace of the ClusterRole is omitted since ClusterRoles are not namespaced.
This ClusterRole can be assigned to Service Account via ClusterRoleBinding
. So that Service Account can list
, get
, watch
pods in the cluster(in all namespaces).
Now I can use the Service Account token(which extracted from the Kubernetes Secret in previous step) as the bearer token with the Authorization header of the request.
Reference
- https://medium.com/rahasak/kubernetes-role-base-access-control-with-service-account-e4c65e3f25cc
- https://iximiuz.com/en/posts/kubernetes-api-call-simple-http-client/
- https://kinvolk.io/blog/2019/02/abusing-kubernetes-api-server-proxying/
- https://labs.f-secure.com/blog/attacking-kubernetes-through-kubelet/
- https://banzaicloud.com/blog/proxy-websocket-kubernetes-api-server/
- https://sysdig.com/blog/monitor-kubernetes-api-server/