The Karpenter Effect: Redefining Our Kubernetes Operations
A reflection on our journey towards AWS Karpenter, improving our Upgrades, Flexibility, and Cost-Efficiency in a 2,000+ Nodes Fleet
For quite some time, at Adevinta, we’ve been hyped by news about AWS Karpenter and its potential to solve some of our most pressing operational challenges.
Now that we’ve fully migrated from Amazon EKS Managed Node Groups to Karpenter, this is the perfect moment to reflect on our journey and share the insights we’ve gained along the way.
The Problem
Managing a fleet of over 2,000 Kubernetes nodes and 30 clusters across 25 marketplaces is no small feat. While using Kubernetes Cluster Autoscaler and Amazon EKS Managed Node Groups served us well initially, we began encountering operational hurdles that hampered our efficiency and scalability.
The complexities of cluster upgrades, the rigidity in instance type selection and limitations in use-case flexibility were becoming increasingly burdensome. We needed a solution that could address these challenges head-on.
Enter Karpenter
Karpenter is an open-source, high-performance Kubernetes node management tool developed by AWS. Unlike a traditional autoscaler, Karpenter dynamically provisions the right compute resources to handle cluster workloads in real-time. It does this by observing the aggregate resource requests of unscheduled pods and making informed decisions to launch new nodes that precisely fit those needs.
Cluster Upgrades and Maintenance Made Seamless
Previously, upgrading our Kubernetes clusters was a stressful process, especially with EKS-managed node groups and provisioning via AWS CDK. The tight coupling between the control plane and node group upgrades made the process fragile. Any issue, like configuration errors or instance shortages, could cause failed stack upgrades and rollback loops.
The challenge was even greater with larger clusters. Upgrading hundreds of nodes while minimising impact could take days, requiring engineers to monitor the process closely. This made upgrades not only time-consuming but also resource-intensive, with failures often caused by external factors such as instance availability.
To mitigate these risks, we had to implement capacity reservations before executing Kubernetes upgrades. However, this approach was inefficient and lacked scalability.
With Karpenter, the process has become much simpler. Control plane and node pool upgrades are now decoupled, making control plane upgrades much more straightforward as most are completed within 15-30 minutes. Karpenter asynchronously manages worker node upgrades, meaning that when the control plane version is updated, Karpenter detects the change in the AMI (Amazon Machine Image) and identifies which nodes need upgrading by marking them as "drifted."
{"level":"INFO","EC2NodeClass":{"name":"default"},"parameter":"/aws/service/eks/optimized-ami/1.30/amazon-linux-2-arm64/recommended/image_id","value":"ami-0d494a2874a2e7ec1"}
{"level":"INFO","EC2NodeClass":{"name":"default"},"namespace":"","name":"default","reconcileID":"b99dbc2a-5112-4121-ab64-7e512bb0399f","parameter":"/aws/service/eks/optimized-ami/1.30/amazon-linux-2-gpu/recommended/image_id","value":"ami-01389330bfd276054"}Drifted nodes will be gradually replaced by the newer version without manual intervention.
This separation has drastically reduced our upgrade efforts and eliminated the need for constant monitoring. For a deeper dive into our previous upgrade challenges, you might find these earlier posts insightful:
- Unpacking the Complexities of Kubernetes Upgrades: Beyond the One-Click Update
- Unpacking the Complexities of Kubernetes Upgrades: Beyond the One-Click Update (Part II)
Fine-Grained Control Over Disruptions
Karpenter’s ability to respect Pod Disruption Budgets (PDBs) has been a significant improvement for us. Previously, the Cluster Autoscaler — even with the serialised option — would force-remove nodes after a short timeout, often leading to service disruptions. Now, however, Karpenter works patiently within the disruption limits we’ve defined.
We’ve experienced issues with managed node groups, such as renaming node groups or adjusting the minimum number of replicas, which caused more disruptions than expected. These situations often made upgrades noisy and led to incidents we had to resolve, ultimately affecting customer uptime. This unpredictability discouraged us from performing upgrades, as they usually resulted in unplanned outages and additional workload.
With Karpenter, we now have far greater control over how we manage node disruptions during upgrades. One of the key benefits is the ability to configure Disruption Budgets to precisely manage the rate of node updates. For instance, we can set a policy that allows a single node to be updated every 15 minutes, significantly reducing the potential impact on running services. This gradual, controlled approach ensures that services remain stable and minimises downtime during upgrades, which was previously a major challenge.
Another advantage is the flexibility Karpenter provides in configuring disruption budgets for different node pool types. We can now assign varying levels of reliability based on workload type. For mission-critical workloads, we enforce stricter limits on node disruptions, while less critical workloads have more relaxed settings, ensuring optimal resource management and service reliability across the cluster.
This disruption budget works hand in hand with full respect to the PodDisruptionBudget (PDB) that the customer configures. For example, if during the 15-minute window, no node could be disrupted at all due to the PDB requirement, the window will be skipped. This ensures high reliability and full compliance with customers’ reliability requirements which is very important in a multi-tenancy scenario.
Moreover, in v1, Karpenter allows us to specify different timeframes for various disruption causes, such as consolidations, expired nodes or spec changes. This granularity helps us maintain high availability while still performing necessary maintenance.
apiVersion: karpenter.sh/v1
kind: NodePool
metadata:
name: default
spec:
template:
spec:
expireAfter: 720h # 30 * 24h = 720h
disruption:
consolidationPolicy: WhenEmptyOrUnderutilized
budgets:
- nodes: "20%"
reasons:
- "Empty"
- "Drifted"
- nodes: "5"
- nodes: "0"
schedule: "@daily"
duration: 10m
reasons:
- "Underutilized"Karpenter also allows us to use a strategy that we call “double jump.” During a “double jump” we upgrade an EKS control plane for two versions consecutively with only a single cluster rebuild. We do this by configuring Karpenter to allow zero nodes during the control plane upgrade, then remove it once the control plane is upgraded two versions ahead.
disruption:
budgets:
{{- if has .Values.clusterName .Values.featureFlags.blockDisruption }}
- nodes: "0"
{{- end }}Karpenter will drift all the nodes as soon as it detects a new version of Kubernetes. So, if a node is marked as drifted, the replacement node will have the latest AMI that Karpenter has discovered at the time of launch. In this case, the nodes that are at 1.28 AMI will be drifted to AMIs at 1.30.
This simple feature saves our operation time tremendously as a general full fleet upgrade requires several weeks to plan and execute. With a double jump, we prevent unnecessary full-fleet rebuilds even with strict restrictions. It’s always better not to disrupt pods whenever possible to reduce noise for our users.
Flexibility with Instance Types
Previously, selecting instance types was a cumbersome process. Each node group required explicit instance types that are strict with the resource requirement. Using a cloud provider does not mean you have unlimited machines all the time, so there’s an actual risk of instance exhaustion, especially in smaller regions. When this happens, an attempted upgrade will fail.
There’s a concept of secondary instance types in AutoScalingGroup where you can provide a list of substitute types and the node group specification. However, it was a manual and error-prone task with many limitations. For example, they must have the same CPU/memory parity as the main instance type. Managing this at scale was nothing short of a nightmare.
Karpenter abstracts away this complexity by allowing us to define instance requirements rather than instance types. Karpenter automatically selects the most appropriate and cheapest instances available. This flexibility not only simplifies management but also mitigates the risk of instance exhaustion — a real concern when operating at our scale.
Another additional benefit is that when there are better/cheaper/more performant instance types available, we don’t need to keep watching or perform a full fleet rebuild of 2000+ machines anymore. The process happens gradually naturally and automatically.
Tailored Node Pools for Diverse Workloads
One of the key aspects of using Karpenter in our environment has been its ability to facilitate the creation of customised node pools tailored to specific workloads. This flexibility is crucial when managing a diverse ecosystem of applications, each with unique resource requirements and operational characteristics.
Simplified Node Pool Creation with Kubernetes CRDs
Karpenter leverages Kubernetes Custom Resource Definitions (CRDs) to define provisioning behaviour, making node pool creation an intuitive process for those familiar with Kubernetes. By defining Provisioner CRDs, we can specify constraints and preferences such as:
- Instance Types and Families: Select from a wide range of EC2 instance types or families to match workload requirements.
- Node Labels and Taints: Assign labels and taints to nodes to control pod scheduling and ensure workloads land on appropriate nodes.
- Kubelet Configuration: Customise kubelet parameters to optimise node performance for specific applications.
This approach allows us to manage node configurations declaratively, reducing the complexity associated with maintaining separate Auto Scaling Groups or managed node groups for each workload type.
Catering to Specific Workload Requirements
Here are some practical examples of how we’ve utilised Karpenter to accommodate diverse workloads:
1. GPU-Intensive Machine Learning Jobs
Our data science teams frequently run machine learning models that require GPU acceleration. Previously, we had to maintain dedicated GPU node groups, which were often underutilised and costly. With Karpenter, we can define a Provisioner with the following specifications:
- Resource Requirements: Nodes must have GPU capabilities (e.g., NVIDIA Tesla V100).
- Instance Types: Preference g4dn instance families.
- Taints and Labels: Apply taints to ensure only GPU workloads are scheduled on these nodes.
When there’s a pod requesting GPU resources with a specific toleration that satisfies the taint specified by us, Karpenter dynamically provisions a GPU-powered node. Once the workload is completed, the node can be decommissioned if no longer needed. This dynamic scaling optimises resource utilisation and reduces costs, and taints and labels allow workloads to be scheduled on node types matching with the resource needed.
2. Dedicated Nodes for monitoring and logging
Our monitoring stack, including Prometheus and log collectors, benefits from being isolated from other workloads to prevent resource contention, pod churns, and possible noisy neighbour effects. Using Karpenter, we could set up a Provisioner that:
- Computes optimisation: Chooses instance types optimised for memory-intensive workloads (e.g., r5 family).
- Applies specific labels and taints: Ensures only monitoring and logging pods are scheduled on these nodes.
- Custom kubelet configuration: Tweaks settings automatically, such as kubelet max pods and eviction policies.
This isolation has stabilised our monitoring services and reduced the “noisy neighbour” effect, where resource-intensive workloads impact the performance of others on the same node. For example, isolating our Prometheus workloads onto dedicated nodes not only improved performance but also facilitated additional cost savings. You can read more specifics about that particular enhancement in our Prometheus cost-saving blog post.
Additional Features for Enhanced Stability
Features like defining startupTaints are also invaluable. They allow us to prepare essential components—such as IAM agents on the node—before scheduling pods. This significantly reduces intermittent issues caused by applications being scheduled onto nodes where mandatory components aren't yet fully operational. By ensuring nodes are completely ready before accepting workloads, we enhance the overall stability and reliability of our services.
Facilitating Flexibility with Kyverno Policies
To provide this level of flexibility seamlessly to our customers, we employ Kyverno policies. These policies automate the assignment of tolerations to pods based on their node selectors, simplifying the process for users and minimising the risk of misconfigurations.
For example, consider the following Kyverno policy:
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
name: enforce-gpu-taint
spec:
validationFailureAction: Enforce
background: false
rules:
- name: enforce-gpu-t4-taint
match:
any:
- resources:
kinds:
- Pod
preconditions:
all:
- key: "{{ request.object.spec.nodeSelector.\"alpha.gpu.node.x.io/nvidia-gpu-name\" || '' }}"
operator: Equals
value: "t4"
mutate:
patchesJson6902: |-
- path: "/spec/tolerations/-"
op: add
value:
key: "alpha.gpu.node.schip.io/nvidia-gpu-name"
operator: "Equal"
value: "t4"
effect: "NoSchedule"
- name: enforce-gpu-a10g-taint
match:
any:
- resources:
kinds:
- Pod
preconditions:
all:
- key: "{{ request.object.spec.nodeSelector.\"alpha.gpu.node.x.io/nvidia-gpu-name\" || '' }}"
operator: Equals
value: "a10g"
mutate:
patchesJson6902: |-
- path: "/spec/tolerations/-"
op: add
value:
key: "alpha.gpu.node.schip.io/nvidia-gpu-name"
operator: "Equal"
value: "a10g"
effect: "NoSchedule"
- name: enforce-gpu-taint
match:
any:
- resources:
kinds:
- Pod
preconditions:
all:
- key: "{{ request.object.spec.nodeSelector.\"accelerator.node.x.io/gpu\" || '' }}"
operator: Equals
value: "true"
mutate:
patchesJson6902: |-
- path: "/spec/tolerations/-"
op: add
value:
key: "accelerator.node.schip.io/gpu"
operator: "Exists"
effect: "NoSchedule"Benefits and Challenges
Kyverno has delivered some significant advantages:
- Simplified User Experience: With Kyverno automating the assignment of tolerations based on node selectors, our customers can deploy their workloads without needing deep expertise in Kubernetes scheduling intricacies. They simply specify their requirements, and the policies handle the rest.
- Consistent and Correct Scheduling: Automating tolerations ensures that pods are scheduled onto the correct nodes with the appropriate taints. This maintains the isolation and performance optimisations we have already established, reducing the risk of pods ending up on unsuitable nodes.
- Reduced Misconfiguration Risks: By handling tolerations automatically, we minimise the chance of human error, such as forgetting to include necessary tolerations, which could lead to pods remaining unscheduled or being scheduled incorrectly.
By integrating Kyverno policies with Karpenter's provisioning capabilities, we've empowered our customers to leverage advanced Kubernetes features effortlessly. This synergy enhances the user experience, optimises resource utilisation and maintains the high performance and reliability standards that our operations demand.
Automated Security Updates
Security is paramount, and keeping our Amazon Machine Images (AMIs) up to date was previously a manual and time-consuming process. With Karpenter, AMI updates are handled seamlessly. When a new AMI is released, Karpenter identifies drifted nodes and gradually replaces them, all while respecting PDBs and disruption policies. This automation ensures our fleet is always up to date with the latest security patches without the overhead of manual intervention.
In the past, we were not actively updating our AMIs as we could not constantly check for updates. Our previous architecture would also require a full fleet rebuild with every update, creating more disruption.
Significant Cost Savings Through Instance Optimisation
One of the key objectives of our team this year has been cost optimisation. We identified that migrating from Intel to AMD instances could save us up to €30,000 per month without compromising performance.
However, executing this migration manually across thousands of nodes was impractical.
Karpenter made this transition effortless. By selecting the most cost-effective instances that met our requirements, Karpenter naturally favoured AMD instances over Intel, creating immediate cost savings.
This benefit was realised as part of our migration to Karpenter, showcasing its ability to deliver both operational efficiency and financial benefits without any additional intervention.
Enhanced Metrics and Insights
Karpenter provides a wealth of metrics that were previously difficult to obtain. From drift detection and node disruption counts to consolidation events and pod scheduling times, these insights have been invaluable. They allow us to make data-driven decisions and proactively address potential issues before they escalate.
While using managed node groups, We only have the metrics about nodes through AutoScalingGroups and those metrics are not native to the cluster. So, the metrics we had from it were not very helpful to understand the behaviour and the movement of nodes during events such as an upgrade or consolidation.
The Challenges We Faced
No journey is without its hurdles, and ours was no exception.
The Importance of Proper Pod Disruption Budget (PDB) Configuration
While Karpenter enhances reliability by respecting Pod Disruption Budgets (PDBs), this benefit depends on customers correctly setting up their PDBs. Misconfigurations can introduce risks:
Risk 1: Increased Disruptions Without PDBs
If customers don't set PDBs, Karpenter may disrupt multiple pods simultaneously during operations like node consolidation. Given Karpenter's frequent optimisations, this could lead to more service disruptions.
Risk 2: Overly Strict PDBs Hindering Maintenance
Conversely, if customers set overly strict PDBs—such as specifying maxUnavailable: 0—they prevent any voluntary pod evictions, including essential node drains for maintenance. This can block necessary operations and degrade cluster health.
Our Mitigation Strategies
To balance reliability with operational flexibility, we've implemented:
- Setting Sane Default PDBs
We provide default PDB configurations for applications that lack them, ensuring protection against disruptions without being overly restrictive. This helps prevent unintended service impacts during maintenance while also permitting necessary node disruptions when required. - Enforcing Policies with Kyverno
Using Kyverno, we enforce policies to prevent PDB configurations that could impede cluster maintenance:
For example:
Preventing Duplicate PDB Selectors
We disallow creating or updating a PDB if another PDB with the same selector exists in the namespace. This avoids conflicts and ensures PDB effectiveness.
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
name: forbid-duplicate-pdb-selectors
spec:
validationFailureAction: Enforce
background: false
rules:
- name: check-duplicate-pdb
match:
any:
- resources:
kinds:
- PodDisruptionBudget
operations:
- CREATE
- UPDATE
context:
- name: existingPDBs
apiCall:
urlPath: /apis/policy/v1/namespaces/{{request.namespace}}/poddisruptionbudgets
jmesPath: "items[?to_string(@.spec.selector.matchLabels) == to_string(`{{ request.object.spec.selector.matchLabels }}`) && @.metadata.name != '{{ request.object.metadata.name }}']"
validate:
message: "A PodDisruptionBudget with the same selector already exists."
deny:
conditions:
any:
- key: "{{ existingPDBs | length(@) }}"
operator: GreaterThan
value: 0Ensuring Reasonable maxUnavailable Values
We enforce that if a PDB specifies maxUnavailable, it must be greater than zero. Setting it to zero prevents all voluntary evictions, hindering maintenance tasks.
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
name: pdb-maxunavailable
annotations:
policies.kyverno.io/title: PodDisruptionBudget maxUnavailable Non-Zero
policies.kyverno.io/subject: PodDisruptionBudget
policies.kyverno.io/description: >-
A PodDisruptionBudget which sets its maxUnavailable value to zero prevents
all voluntary evictions including Node drains which may impact maintenance tasks.
This policy enforces that if a PodDisruptionBudget specifies the maxUnavailable field
it must be greater than zero.
spec:
validationFailureAction: Enforce
background: false
rules:
- name: pdb-maxunavailable
match:
any:
- resources:
kinds:
- PodDisruptionBudget
preconditions:
any:
- key: '{{ regex_match(''^[0-9]+$'', ''{{ request.object.spec.maxUnavailable || ''''}}'') }}'
operator: Equals
value: true
validate:
message: "The maxUnavailable field, when specified as an integer, must be greater than zero."
pattern:
spec:
=(maxUnavailable): ">0"Restricting the karpenter.sh/do-not-disrupt Annotation
We prevent users from adding the karpenter.sh/do-not-disrupt annotation to workloads without proper justification, as overuse can impede necessary maintenance.
By implementing these measures, we maintain a balance between application reliability and the ability to perform essential cluster operations.
Over-provisioning and Scheduling Latency
While Karpenter has significantly improved our resource utilisation and cost efficiency, we’ve encountered challenges related to scheduling latency, particularly during periods of rapid scaling. The optimisation and densification of nodes under Karpenter means that resources are used more efficiently, but this can inadvertently slow down the scaling response time for our customers.
The Challenge of Scheduling Latency
In our high-scale environment, we frequently experience bursts of activity that require rapid scaling of pods. With Karpenter’s dense packing of nodes, there are fewer idle resources available at any given time. When a sudden demand spike occurs, there may not be an immediate capacity to schedule new pods, resulting in them entering a pending state while Karpenter provisions additional nodes.
This latency is further compounded by our use of a startupTaints. When Karpenter launches a new node, we apply a taint that prevents pods from being scheduled on it until our in-house Kubernetes operator verifies the node's status and removes the taint. This verification step is crucial for ensuring node health and compliance with our operational policies. Unfortunately, it also introduces additional delay in making the new capacity available.
As a result, we’ve observed an increase in our pod scheduling time Service Level Indicator (SLI), and customers have reported performance impacts due to the delayed scaling.
Implementing Over-Provisioning with Low-Priority Pods
To mitigate these scheduling delays, we’ve adopted an over-provisioning strategy similar to the approach described in the Cluster Overprovisioner by codecentric and the Karpenter Overprovisioning Blueprint.
Here’s how we’ve implemented it:
- Creating a Low-Priority Class: We defined a custom Kubernetes PriorityClass with a lower priority than any of our standard workloads. This class is used to schedule placeholder pods that can be preempted when higher-priority pods need to be scheduled.
- Deploying Placeholder Pods: We deploy a set of pods using this low-priority class. These pods request minimal resources but are sufficient to keep nodes provisioned and ready. They act as “scapegoats” or buffer workloads that occupy capacity without significant resource consumption.
- Facilitating Immediate Scheduling for High-Priority Pods: When new, higher-priority pods are scheduled, they preempt the low-priority placeholder pods. The scheduler evicts the placeholder pods, freeing up resources on existing nodes without waiting for new nodes to be provisioned. This allows the high-priority pods to be scheduled immediately, reducing latency.
- Dynamic Node Provisioning: Karpenter continues to monitor the cluster state and recognises that the eviction of placeholder pods has reduced the buffer capacity. It then provisions new nodes to maintain the over-provisioned state, scheduling new placeholder pods onto them.
apiVersion: scheduling.k8s.io/v1
kind: PriorityClass
metadata:
name: overprovisioning
value: -1
globalDefault: false
description: "Priority class for overprovisioning pods."
apiVersion: apps/v1
kind: Deployment
metadata:
name: overprovisioning-placeholder
spec:
replicas: 10
selector:
matchLabels:
app: overprovisioning-placeholder
template:
metadata:
labels:
app: overprovisioning-placeholder
spec:
priorityClassName: overprovisioning
containers:
- name: pause
image: k8s.gcr.io/pause:3.2
resources:
requests:
cpu: X
memory: XFuture Improvements
We’re actively exploring enhancements to our current approach:
- Dynamic Adjustment of Over-Provisioned Capacity: Implementing automation that adjusts the number of placeholder pods based on historical usage patterns and predictive scaling, further optimising resource utilisation. As our cluster size varies, the current approach can be too expensive for a small cluster.
- Community Collaboration: We’re actively subscribing to this mega issue in the Karpenter upstream where they’re discussing how to over-provision resources natively via Karpenter.
The “Cheapest First” Pitfall
While Karpenter’s cost-optimisation is generally beneficial, it’s not always ideal. We discovered that for example, the m5a instances were marginally cheaper than m6a instances:
- m5a.4xlarge in eu-west-1: $0.7680/hr
- m6a.4xlarge in eu-west-1: $0.7704/hr
Despite the negligible price difference, m6a instances offer significantly better performance. Karpenter’s default behaviour to select the cheapest instance led to suboptimal performance for some CPU-intensive applications, causing them to scale out unnecessarily and, paradoxically, increase costs.
Our customer reports a huge increase in the replica numbers of their CPU-intensive applications after the Karpenter migration even without any code change.
It’s not only the number of replicas that’s a problem, but also the latency of their service increases extensively despite no changes being made to the code.
From our investigation, we found that as m5a is cheaper it becomes the new Karpenter preference replacing previous m6a instances.
We resolved this challenge by defining new node pools with higher weight (priority) that prefer 6th-generation instances and relegating 5th-generation instances to a fallback option.
- key: karpenter.k8s.aws/instance-generation
operator: Gt
values:
- "5"
weight: 15
- key: karpenter.k8s.aws/instance-generation
operator: Gt
values:
- "4"
weight: 10This adjustment ensured optimal performance and cost-efficiency while making sure that if the 6th generation allocation is exhausted we’ll still have the node pool to absorb workloads.
I have also opened the discussion with the maintainer to ask if it makes sense to have a native solution to this scenario.
Compatibility Handling During v1 Upgrade
Upgrading to Karpenter from v0.37.x to v1 presented some challenges. The conversion process and backward compatibility weren’t as smooth as anticipated, leading to confusion. While these issues were manageable, they highlighted the need for caution and thorough testing when adopting new versions, especially for using ArgoCD to manage the Karpenter installation.
Here are just a few of the issues you may encounter during a Karpenter upgrade:
Kubelet configuration not applied after v1 upgrade · Issue #6987 · aws/karpenter-provider-aws
Karpenter conversion webhooks do not work on chart version 1.0.2 · Issue #6982 ·…
Karpenter v1.0.0 chart does not respect Helm values in packaged CRDs leading to ArgoCD upgrade…
Looking Ahead
Our journey with Karpenter is far from over. Here are some areas we’re exploring:
Migrating Karpenter Controller Off Managed Node Groups
We currently run the Karpenter controller on a managed node group dedicated to control-plane components. However, we’re considering moving it to AWS Fargate as some teams in the industry already do. This transition would eliminate the need for managed node groups, further simplifying our infrastructure.
Enhancing Over-Provisioning Capabilities
We’re closely following discussions within the Karpenter community about introducing capacity reservation and over-provisioning features. In the meantime, we’re investing in refining our over-provisioning tools to be more flexible and less reliant on hardcoded configurations.
Conclusion
Migrating to AWS Karpenter has been a transformative experience. We’ve achieved smoother cluster upgrades, greater flexibility in instance selection, improved workload isolation, automated security updates and significant cost savings. While challenges remain, the benefits have far outweighed the drawbacks.
For organisations managing Kubernetes at scale, Karpenter offers compelling advantages that can streamline operations and reduce overhead. As we continue to refine our use of Karpenter, we look forward to sharing more insights and contributing to the community’s collective knowledge.
We hope our experiences provide valuable insights for your own Kubernetes journey. Feel free to share your thoughts or questions in the comments below.
