Get shell access to pods & nodes in Kubernetes using kubectl

Accessing pods and nodes in a Kubernetes cluster can save you a lot of time debugging issues. Shell access is your only way to activate tools within the container/pod/node without planning that in advance.

So whether your pods are crashing or nodes are malfunctioning and you are looking for a way to deep dive and don’t have a clue how — you are in the right place.

Shell is fast, reliable, powerful, and simple. All the knowledge and shell tools that have been developed and gathered are available to save plenty of time debugging issues. Having powerful tools by your side while your deep-diving into an issue can make the difference between a 5 minutes issue to a number of hours or even days issue.

Here are the easiest and most useful ways to get it :

1. Open shell in a running pod/container
2. Open a shell to a node using kubectl
3. Attach debug container to a running pod and open a shell to it
4. Duplicate a pod to a sandbox and open a shell to it

1. Open shell in a running pod/container

In case you want to open a shell to the container, you can use this command. You don’t need SSH network connectivity, kubectl will proxy your terminal directly to the container.

kubectl exec -it <pod-name> —- /bin/sh
kubectl exec -it <pod-name> -c <container-name> —- /bin/sh

Full example:

What can you do next?

Now you are in a running container inside the pod, you can:

• Access other logs which are not directed to stdout
• Test your application (like using curl)
• Restart the process inside the pod
• Run a diagnostic script that you packed up in the image
• Run the same process with different parameters (debug mode)
• and just almost anything you like…

Pay attention that you can use only the commands that are available in the container image, if this container has a light image like alpine, many commands will not be available, sometimes even the bash shell — to extend the available commands jump to sections 3 & 4.

Like any other powerful tool, you need to be careful too when using it, killing a process or restarting it can cause damage, for example, killing PID 1 can cause the container to restart and the shell to disconnect.

2. Open a shell to a node using kubectl

In case you want to access a node to check what is going on there, you can use this command. Here you need to pick the node name and the image.

kubectl debug node/<node-name> -it --image=<image name>

Full Example:

What can you do next?

Debug the node, there are plenty of options here, like:

check the node configuration files

cat /host/var/lib/kubelet/config.yaml

check the kubelet logs:

1. chroot to the host to get access to host commands

chroot /host

2. get the kubelet logs

journalctl -u kubelet

Basically, after chrooting, you can use many host commands :-)

3. Attach debug container to a running pod and open a shell to it

What about magic stuff? In the previous pod command, we used only the capabilities that are already available in the pod. With an ephemeral container, we can introduce new commands and capabilities to a running pod. Choose the right image and add powerful tools to debug easier!

kubectl debug -it pods/<podname> --image=<image-name> -- <command>

Full example:

What can you do next?

Use the tools in the ephemeral container to debug the running containers/pods. for example, you can curl from the container to the pod to make sure your service is running properly, for example:

kubectl debug -it pods/service-b-774d4f6cd4-bjf85 -image=ellerbrock/alpine-bash-curl-ssl -— curl 127.0.0.1:8080

FYI: The ephemeral containers feature gate must be turned on to use this command.

4. Duplicate a pod to a sandbox and open a shell to it

When we can’t or don’t want to debug on the running pod, we would like to debug it in a sandbox environment in which traffic is not redirected to it. For example, debug the problematic pod while reverting the service. The following command allows you to copy the container and attach a debug container to it.
This does not require the ephemeral containers feature to be available!

Command:

kubectl debug pod/<pod-name> -it --image=<image-name> --copy-to=my-debugger

Full example:

What can you do next?

like option 1/3, you can access and test the pod accessibility.

Don’t forget to delete the pod

The debug pod will not be deleted automatically so it is required to delete it manually :

kubectl delete pod <podname>

Let’s connect — https://linktr.ee/the_good_guy