Nerd For Tech
Published in

Nerd For Tech

Photo by John Barkiple on Unsplash

EKS Networking— VPC CNI Modes

Pod IP addressing in EKS

AWS EKS uses a simple approach for pod networking. It uses the IPs from the VPC for the Pod. The same is achieved by leveraging the VPC CNI plugin, a daemonset(aw-node) running in the cluster. VPC CNI plugin acts as a Bridge between K8s & AWS VPC.

This plugin leverages the fact that EC2 instances can have multiple network interfaces (ENI) and each ENI can have more than one IPs. The CNI plugin provisions secondary interfaces and adds multiple IPs to it. When a pod creation request comes it allocates an IP from the pool of IPs attached to secondary ENI. When the number of pods running on the node exceeds the number of addresses on a single ENI, the CNI backend starts allocating a new ENI. CNI Plugin exposes the below-mentioned configurations to control IP & ENI reservation.

WARM_ENI_TARGET
WARM_IP_TARGET
MINIMUM_IP_TARGET

To understand the above configuration visit my earlier article https://medium.com/nerd-for-tech/eks-networking-cni-457ae298b9e6

The plugin runs mainly in three modes.

IP Mode

In this mode, the secondary ENI reserves the IP address from the subnet.

Image by author

Prefix Mode

In this mode, the secondary ENI maps to a subnet prefix and allocates IP to pods from the prefix. The prefix is of /28. This means we are increasing the number of IPs available to nodes by a factor of 16. Say for c5.large we can have 3 ENI and each can have 10 IPs. This gives us 27 usable IPs. But with prefixes, we will have 432 (3*9*16) IPs. This allows us in greater pod density.

Image by author

Custom configuration mode

In this mode, CNI will use secondary CIDRs. Secondary CIDR ranges are an additional set of subnets deployed to the same availability zones as your existing private subnets. The only difference, these subnets use a secondary CIDR range, rather than the primary CIDR range attached to the VPC. In this mode, the worker node will get IP out of our primary CIDR range and pods will get IP out of our secondary CIDR.

Image by author

In this blog, I tried to explain how IPs are assigned to Pods in EKS in different modes.

Happy networking!!

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Amit Singh Rathore

Amit Singh Rathore

1.4K Followers

Staff Data Engineer @ Visa — Writes about Cloud | Big Data | ML