How to TCPdump effectively in Kubernetes (part 1)

Philippe Bogaerts
May 9, 2019 · 2 min read

In a previous blog post, we focused on how to TCPdump in docker containers (see https://dockersec.io/@xxradar/how-to-tcpdump-effectively-in-docker-2ed0a09b5406).

Although the information is still very useful and valid for troubleshooting K8S pods, it might get more difficult figuring out which containers to attach to on what node, etc … but a very valid approach.

While focusing on an easier way, I came across the command

kubectl patch 

This command allows to update a deployment for example. It basically does the trick outlined in the previous post but fully automatic

So if you have deployment running like for example

kubectl create -f https://raw.githubusercontent.com/xxradar/kuberneteslearning/master/radarhack-deploy.yamlkubectl create -f https://raw.githubusercontent.com/xxradar/kuberneteslearning/master/radarhack-expose-clusterip.yaml

You should be able to access it on

# kubectl get services
...
my-radarhack-clusterip ClusterIP 10.104.201.226 <none> 80/TCP 36d
# curl http://10.104.201.226/
<HTML>
<HEAD>
<TITLE>RADARHACK.COM by XXRADAR</TITLE>
...

So far so good. Let’s focus on how to add the TCPdump container to the deployment. Create following file ex. patch.yaml

spec:
template:
spec:
containers:
- name: tcpdumper
image: docker.io/dockersec/tcpdump

And apply it

kubectl patch deployment radarhack-deployment --patch “$(cat patch.yaml)”

You should be able to see that the TCPdump container is automatically added to the pods (please note that the pods are recreated, which is not exactly the same as in the previous blogpost, where you connect to a running pod/container)

# kubectl get deployment radarhack-deployment  --output yaml
apiVersion: extensions/v1beta1
kind: Deployment
..
labels:
app: radarhack
spec:
containers:
- image: docker.io/dockersec/tcpdump
imagePullPolicy: Always
name: tcpdumper
resources: {}
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File

- image: docker.io/xxradar/naxsi5
imagePullPolicy: Always
name: radarhack
ports:
- containerPort: 80
protocol: TCP
resources: {}
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
dnsPolicy: ClusterFirst
restartPolicy: Always
schedulerName: default-scheduler
securityContext: {}
terminationGracePeriodSeconds: 30
status:
availableReplicas: 3
conditions:
...

Now you can attach to the pod (and if traffic is generated)

# kubectl get pod
NAME READY STATUS RESTARTS AGE
...
radarhack-deployment-7c6b8f595-85dkt 3/3 Running 0 33m
radarhack-deployment-7c6b8f595-8qffp 3/3 Running 0 33m
radarhack-deployment-7c6b8f595-grdxz 3/3 Running 0 33m
radarhack22-56b59f7c7c-7cx2w 1/1 Running 0 36d
radarhack22-56b59f7c7c-bsnjk 1/1 Running 0 36d

# kubectl attach -it radarhack-deployment-7c6b8f595–85dkt
Defaulting container name to tcpdumper.
Use ‘kubectl describe pod/ -n default’ to see all of the containers in this pod.
Unable to use a TTY — container tcpdumper did not allocate one
If you don’t see a command prompt, try pressing enter.
l 3777658557 ecr 3777667637], length 0
08:14:38.740879 IP 10.244.0.0.48528 > radarhack-deployment-7c6b8f595–85dkt.80: Flags [.], ack 7945, win 360, options [nop,nop,TS val 3777667637 ecr 3777658557], length 0
08:14:38.936447 ARP, Request who-has radarhack-deployment-7c6b8f595–85dkt tell 10.244.2.1, length 28
08:14:38.936473 ARP, Reply radarhack-deployment-7c6b8f595–85dkt is-at fa:7a:24:85:f1:56 (oui Unknown), length 28
08:43:37.932430 IP 10.244.0.0.58290 > 10.244.2.50.80: Flags [S], seq 877954416, win 29200, options [mss 1460,sackOK,TS val 3779406828 ecr 0,nop,wscale 7], length 0
08:43:38.466295 IP 10.244.0.0.58292 > radarhack-deployment-7c6b8f595–85dkt.80: Flags [S], seq 3362221005, win 29200, options [mss 1460,sackOK,TS val 3779407363 ecr 0,nop,wscale 7], length 0
08:43:38.466343 IP radarhack-deployment-7c6b8f595–85dkt.80 > 10.244.0.0.58292: Flags [S.], seq 1653377327, ack 3362221006, win 27960, options [mss 1410,sackOK,TS val 3779398282 ecr 3779407363,nop,wscale 7], length 0

Philippe Bogaerts

Written by

BruCON co-founder, OWASP supporter, AviNetworks presales engineer and Docker enthusiast! Interested in webapp security and pentesting, music and food !!

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade