Unleashing the power of Karpenter
What is karpeneter?
Karpenter is an open-source, flexible, high-performance Kubernetes cluster autoscaler built with AWS. Karpenter improves the efficiency and cost of running workloads on Kubernetes clusters by watching for unschedulable pods, evaluating scheduling constraints, provisioning nodes that meet the requirements and removing the nodes when the nodes are no longer needed. It is now CNCF project, accepted in sigs auto-scaling group.
How Karpenter works?
Before we understand how karpeneter works, let’s look at cluster autoscaler (CA) and auto-scaling groups(ASG). As traffic increases, the pods scale up with horizontal pod autoscaler. Once node reaches its capacity and a new pod is in pending state, CA will interact with ASG to add a new node. Kapenter eliminates the need for CA and ASG. Pending pods talk to karpenter directly, which interacts with EC2 api and spins up new node.
Karpeneter is defined with 2 yaml files:
1. nodepool (formely known as provisioner)
2. EC2nodeclass(formely known as AWSNodeTemplate)
Karpeneter gives the flexibility to define what kind of EC2 instances you need. Different attributes are:
instance type flexilibity
instance family [c5, m5, r5]
instance size [nano, micro, small]
AZ flexilibity
Availability zone: [us-east-1a, us-west-2b]
Architecture flexilibity
architecture [amd64, arm64]
Purchase options flexilibity
capacity-type [spot, on-demand]
It gives you the option to include operators like ‘In’,’NotIn’,’greaterThan’ etc.
if you leave them to default, it will pick the best suit instance type depending on your pod definition.
You can define limits on how many EC2 instances this nodepool can provision.
Nodepool strategies
workloads may be required to run on certain AZs or certain instance types like graviton or purchase options like on-demand or spot
This can be achieved using the Standard K8s scheduling mechanisms
node selectors
node affinity
taints and tolreations
topology spread
Karpneter adds label for the keys defined in the nodepool yaml. These labels can be used in the deployment or pod definition to run workloads on those required specific nodes.
There are 3 major stragtegies for nodepools:
1. single
A single nodepool can manage compues for mutiple teams and workloads
use case: dingle nodepool for graviton and x86, while pod has requirement for specific processor type
2. mutiple
isolating compute for different purposes
use case: security isolation, team seperation
3. weighted
Defined across youe nodepools so that node scheduler will atempt to schedule with one nodepool before other
use case: prioritse RI and savings plan before other instance types.
Benefits of karpeneter:
Cost optimzation
Karpenter not only helps with scaling out but also scale in when a node is under utilized. It monitors the usgae of node and if required can dwongrade the node size for optimal utilizaion.
Simplifies dataplane management
Karpenter does work of 4 different resources:
cluster autoscaler
node groups
node termination handler
descheduler
This simplifies the dataplane management, therefore reducing operational overhead.
