Understanding Kubernetes Networking — Part 1

This article will be a start for series of posts describing how Kubernetes Networking work.

We are now, dealing with more and more environment where there are Kubernetes deployment in play and we should be aware as to where we can perform a packet-capture or look for results in containerized environment.

• Here, I am assuming that we are aware of the basic terminology related with Kubernetes — like; containers, POD, nodes, kubelet etc.
• We will be explaining some new terminologies along the way.
• Ubuntu 16.04 are nodes’ s OS. It is Azure Kubernetes (AKS) deployment.

Let’s get started.

• Kubernetes is an orchestration tool to manage your containerized application.
• There are 4 distinct issue to be address in Kubernetes Networking:

1.) Container to Container communication.

2.) POD to POD communication.

3.) POD to Service communication.

4.) External to Service communication.

• The fundamental requirement for networking that Kubernetes imposes are:

- Pods on a node can communicate with all pods on all nodes without NAT.

- Agents on a node (e.g. system daemons, kubelet) can communicate with all pods on that node.

- Pods in the host network of a node can communicate with all pods on all nodes without NAT.

Now, keeping these fundamentals in check, we will explore how we can solve all the 4 networking issues of Kubernetes.

More information: https://kubernetes.io/docs/concepts/cluster-administration/networking/#the-kubernetes-network-model

1. Container to Container communication.

• We usually have just one container inside a POD.
• But we can have more than one container inside a POD. This is done because the other containers can help your primary application container.
• These helper containers are usually data puller or pushers or proxies.
• Container to Container communication can take place via 2 ways:

Shared volume.

Localhost [Loopback interface].

• PODs are assigned IP addresses.
• Containers in same POD have same IP address, same storage, and network namespace.

We created a POD named — busybox, which has 3 containers deployed in it [busybox1, busybox2 and busybox3] .

• If we get shell into each container and check for their IP addresses, we will observe that they have the same IP address as that assigned to the POD.

What are Namespaces?

• A concept in Linux for isolating resources like network, mount etc. on a machine.
• So, one process will see one set of resources available for it and another process will see a different resource available for it.

There are total 8 types of Namespaces:

• Cgroup
• IPC
• Network
• Mount
• PID
• Time
• User
• UTS

Note: Time namespace is recently added.

• Network namespace virtualizes the network stack.
• Each network namespace will have a private set of IP addresses, its own routing table, socket listing, connection tracking table, firewall, and other network-related resources.
• You can check them under /proc/<pid>/ns

• Here process 5850 is in a different “net” namespace then process 4740 .
• Hence both these processes will their own set of routing, firewall rules etc.
• This is the reasons why we can run different containers under different context.

More information:

• https://man7.org/linux/man-pages/man7/namespaces.7.html
• https://en.wikipedia.org/wiki/Linux_namespaces

apiVersion: v1
kind: Pod
metadata:
  name: two-containers
spec: 
   restartPolicy: Never
   volumes: 
     - name: shared-data
       emptyDir: {}
   containers:
       - name: nginx-container
         image: nginx
         volumeMounts:
           - name: shared-data
         mountPath: /usr/share/nginx/html
       - name: debian-container
         image: debian
         volumeMounts:
           - name: shared-data
               mountPath: /pod-data
               command: ["/bin/sh"]
               args: ["-c", "echo Hello from the debian container > /pod-data/index.html"]

• We will deploy 2 containers in same POD and see how we can communicate via shared volume and localhost.
• nginx-container and debian-container
• The shared path between them is: /usr/share/nginx/html (for nginx) /pod-data (for debian).
• So, an update made in /pod-data/index.html, can be seen in /usr/share/nginx/html/index.html

• If we check the “localhost”, we can see that it is the change made by “debian-container” reflecting in “nginx-container” .

• In the same manner, if we make change on “nginx-container” /index.html file and try to perform curl on the “localhost” of some other container (inside same POD), we would see the nginx-container’s output.

apiVersion: v1
kind: Pod
metadata:
 name: new-containers
spec:
   restartPolicy: Never
   
   containers:
      - name: nginx-container
        image: nginx
      
      - name: curl
        image: curlimages/curl
        command: [ "sleep", "600" ]

Nginx output from “curl” POD

and, this brings us to the end of our first post. :)

In the next post we will discuss on POD to POD communication.