Building Amazon EKS Clusters with Graviton2 Instances using Terraform
Last week AWS announced the general availability of AWS Graviton2 instances for Amazon Elastic Kubernetes Services (EKS), allowing the deployment of production workloads using Arm-based instances. At the time of writing, the newest generation of EC2 instances powered by Graviton2 Arm processors deliver up to 40% better price/performance over comparable current generation x86-based instances. This can provide EKS users significant cost-savings for workloads that can take advantage of the Arm processor architecture.
TLDR: You can now use Graviton2 instances with EKS, and doing so may save you money. If you’re running any reasonable number of EC2 nodes for your EKS clusters you should consider looking in to the benefits.
In this article we’ll take a look at building an EKS cluster using Terraform that takes advantage of Graviton2 EC2 instances by way of EKS Managed Node Groups. It assumes working knowledge of AWS, EKS and Terraform.
A fully working example can be found on GitHub.
Why Terraform?
There are many ways to build an EKS cluster, such as the AWS Console, eksctl, CloudFormation and AWS CDK. Many of these are either maintained by AWS or otherwise promoted (like eksctl), so support already exists along with articles for how to leverage it in various forms.
In this example we’re going to use Terraform to showcase a contribution I made to the Terraform AWS Provider that’s been released with version 3.3.0 that allows use of Graviton2 instances with EKS Managed Node Groups.
Arm Workloads on Kubernetes
Before jumping in, its worth quickly discussing the concept of running Arm-based workloads on Kubernetes.
Kubernetes was designed to allow its worker nodes to be made up of heterogeneous machines (virtual or otherwise), meaning that each node can be configured differently. Not only does this provide flexibility in terms of provisioning capacity, it also allows for infrastructure to be provided to fulfill specific use-cases. Common examples of this are nodes that can provide GPU capabilities, or nodes that have more proportional processing power for CPU-bound workloads.
Although from one perspective Arm nodes are simply providing another hardware configuration on which Kubernetes can schedule workloads, there is some additional complexity introducing a new processor architecture. Until recently, the majority of instance types provided by the main cloud vendors were almost exclusively based on x86 processor architectures. Depending on the technology stack of your workloads, there will be varying degrees of work required in porting code to run on an Arm architecture, as well as building multi-architecture container images.
The details of this topic are beyond the scope of this article, but AWS have already published a blog post that provides more insight, as well as an example.
Building the Cluster
With the support added to the Terraform AWS provider, building a cluster to take advantage of the new Arm-based instances types does not differ significantly from using x86-based instances.
Below is a partial snippet we can review for the relevant configuration. See the linked GitHub repository for a full example.
The key takeaways of this configuration snippet are:
- The AWS provider
versionmust be at least 3.3.0 - The EKS Node Group should use a Graviton2 instance type (in this case
m6g.medium), as must specify theami_typeasAL2_ARM_64.
After using Terraform to apply the configuration in the example GitHub repository, we can inspect the Node Group that was created:
The configuration shows that it was successfully created with the Graviton2 instance type that we specified.
The EC2 Console show an additional level of detail regarding the specific instances that were provisioned by the Node Group:
EKS automatically configures the appropriate Arm-compatible AMI to use for these EC2 instances.
EKS clusters do not need to use either x86 or Arm nodes exclusively, and there would be no issue creating an additional Node Group that uses the former processor architecture to provide a heterogeneous set of instances that can cater for a variety of workloads.
Conclusion
The AWS Graviton2 instance family represents a really exciting new opportunity to save on AWS compute costs, and this capability has now been unlocked for EKS users. The simple example above illustrates how Terraform users who consume EKS can start to build on top of this new configuration option to migrate their workloads to instances built on Arm processors and cut some costs in the monthly cloud bill.
