Accessing the kubernetes Service in the Default Namespace from Your Pods

Ever notices that kubernetes service is located in the default namespace and wondered about its purpose?

If you look into this service, you will get a hint about the use of this service.

First, there is no Selector specified. This means that there are no backend pods associated with this service to respond to requests. Second, the type of this service is ClusterIP, which indicates that its purpose is for internal use only. ClusterIP creates a Virtual IP (VIP) that can only be accessed by workloads within the cluster. In this example, the Virtual IP is 10.43.0.1. Lastly, as you can guess from the label component=apiserver, its target port is 6443, which is the default port of the kube-apiserver. In general, this kubernetes service is used for workloads within the cluster wanting to access the API server.

But how does this work? You can find the answer in the official Kubernetes documentation.

Pods that wish to connect to the API server can do so securely by leveraging a service account so that Kubernetes will automatically inject the public root certificate and a valid bearer token into the pod when it is instantiated. The kubernetes service (in default namespace) is configured with a virtual IP address that is redirected (via kube-proxy) to the HTTPS endpoint on the API server.

Yes, we need to leverage a service account to connect to the API server.

Whenever a pod is instantiated, Kubernetes mounts the Cluster CA certificate and Service account token to the path “/var/run/secrets/kubernetes.io/serviceaccount”. The Cluster CA certificate (ca.crt) is used to verify the SSL certificate of the Kubernetes API server, ensuring secure communication. The Service account token (token) is used for authenticating requests to the API server, allowing the pod to securely interact with the Kubernetes API server. You can check those files by running these two commands:

kubectl run -I --tty --rm debug --image=alpine --restart=Never -- sh
cd /var/run/secrets/kubernetes.io/serviceaccount && ls

You can now try to make a connection with the API server, but you need to download curl first:

apk add --no-cache curl

We know that the Kubernetes service lives in the default namespace, so we can use its domain name to access this service. By running the command below, we can try to make a connection to the API server:

curl --cacert /var/run/secrets/kubernetes.io/serviceaccount/ca.crt \
     -H "Authorization: Bearer $(cat /var/run/secrets/kubernetes.io/serviceaccount/token)" \
     https://kubernetes.default.svc.cluster.local

Or you can use the environment variables automatically configured by Kubernetes for pods to access the API server. Kubernetes sets the KUBERNETES_SERVICE_HOST and KUBERNETES_SERVICE_PORT environment variables in each pod, which can be used to construct the URL for accessing the API server. Here’s an example of how you can use these variables with curl to make a request to the API server:

curl --cacert /var/run/secrets/kubernetes.io/serviceaccount/ca.crt \
     -H "Authorization: Bearer $(cat /var/run/secrets/kubernetes.io/serviceaccount/token)" \
     https://$KUBERNETES_SERVICE_HOST:$KUBERNETES_SERVICE_PORT

However, it returns a Failure Status as shown in the screenshot below:

This means that the service account this pod is using does not have the authority to access the given API path. By default, pods use the service account associated with their namespace, and if not specified in the pod’s manifest, pods will use the default service account in their namespace.

This default service account has no specific authorities given, so for you to use this service account, you need to grant some authorities to it using Role and RoleBinding.

apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  namespace: default
  name: service-reader
rules:
- apiGroups: [""]
  resources: ["services"]
  verbs: ["get", "list"]

apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: service-read-access
  namespace: default
subjects:
- kind: ServiceAccount
  name: default
  namespace: default
roleRef:
  kind: Role
  name: service-reader
  apiGroup: rbac.authorization.k8s.io

kubectl apply -f role.yaml
kubectl apply -f rolebinding.yaml

Since I only gave permission to get and list services within the default namespace, the curl requests should be changed as shown below:

Using DNS name:

curl --cacert /var/run/secrets/kubernetes.io/serviceaccount/ca.crt \
     -H "Authorization: Bearer $(cat /var/run/secrets/kubernetes.io/serviceaccount/token)" \
     https://kubernetes.default.svc.cluster.local/api/v1/namespaces/default/services

Using environment variables:

curl --cacert /var/run/secrets/kubernetes.io/serviceaccount/ca.crt \
     -H "Authorization: Bearer $(cat /var/run/secrets/kubernetes.io/serviceaccount/token)" \
     https://$KUBERNETES_SERVICE_HOST:$KUBERNETES_SERVICE_PORT/api/v1/namespaces/default/services

Now, let’s see the result:

Now, you can read the services in the default namespace from within your pod. If you try to get other resource information, such as Pods, you will encounter the Forbidden Status again, as shown below:

Understanding the use of Kubernetes service in default namespace and how to access the Kubernetes API server from within your pod is crucial for managing and interacting with your Kubernetes cluster efficiently. By leveraging service accounts and configuring appropriate roles and bindings, you can control the access and permissions of your pods, ensuring secure and organized operations within your cluster.

By following these steps, you can effectively manage and interact with the Kubernetes API server from within your pods, enhancing the security and efficiency of your cluster operations. I hope you find this article useful :)