Setting up your first Elastic Kubernetes cluster in AWS
Ok, I'm writing this post because it has been a good goal for me to learn more about how to implement Docker images in AWS, and now, I'm on the Kubernetes level and I want to share it with you.
This post may contain a lot of concepts you should know before getting started with Kubernetes and AWS. I’ll list them here. Also, you can follow this post using some NodeJS API you want to Dockerize and move to Kubernetes. You should know about:
AWS Prerequisites and Knowledge:
- IAM Users by programmatically and Console access.
- IAM Roles for your services.
- AWS Command-line interfaces usage.
- VPC, Availability zone, subnets & Security Groups.
Docker & Kubernetes Prerequisites and knowledge:
- Basic docker usage.
- Docker Desktop & kubectl installed on your machine.
Now, what we’re going to do here is:
- Dockerize an API powered by the Express framework for NodeJS
- Create the K8s Deployment files
- Upload the image to ECR (It’s like docker hub but in AWS) and finally
- Deploy our application in the EKS Cluster.
First, let’s prepare all our environments in AWS to allow creating and managing the clusters without issues. Elastic Kubernetes service needs to be managed only by the IAM user that created the cluster, so, let’s create a new IAM user with Console and programmatic Access.
IAM Resources
Let’s start: Go to AWS IAM Dashboard and click on New User. For this user, we’ll set the programmatic access first. Keeping in mind the name of the user is going to be useful to create the cluster and the Access/Secret keys work for configuring our EKS cluster.
Having the user-created, let’s move on to the next step. To create the cluster and the Group nodes we’ll need some roles created in our Account to give them access to create and manage EC2 instances when EKS needs it.
You need to create exactly 2 new roles:
- EKS Role: Role to assign to the EKS cluster to allow interaction with their nodes and their pods’ internal configuration. When you’re on the IAM Role page, click on create new Role and choose from the services list: EKS, then, EKS Cluster for the use case as bellow:
Continue the wizard as usual and let’s get to the next role to be created:
2. Node Groups Role: The Node groups are basically a group of EC2 or AWS Faragate Services, those will contain your pods and the basic configuration to handle the networking and communicate with all the AWS Services such as VPC and load balancers.
For this service we need to choose an EC2 Service Role and assign the followings permissions:
AmazonEKSWorkerNodePolicyAmazonEC2ContainerRegistryReadOnlyAmazonEKS_CNI_Policy
Each permission allows to EC2 instances create the node to communicate with AWS, pull images from ECR and Assign the right IP inside the VPC and subnet configuration.
VPC And networking configuration
Only for the post purposes we can create our new VPC to provision or cluster with the Wizard we found in the VPC Dashboard Page, let’s move on directly to step 2:
When you’re configuring the VPC options you can leave all the options with the default values and update the VPC Name and Subnet name.
Note: For EKS Requirements, we need to create at least two availability zones linked to this new VPC we just created, for high availability proposes. So, let’s create a quick subnet:
Go directly: https://console.aws.amazon.com/vpc/home?region=us-east-1#CreateSubnet:
This is the dashboard to create the new subnet. First, choose the VPC you want to relate to in the VPC ID field. In this case, is our last VPC created in the previous step.
Now, we need to specify some network options related to the subnet.
- You must assign a different availability zone than the one in the previous step.
- In IPv4 CIDR Block, you can use
10.0.2.0/24 - Finish your subnet.
By default, the subnets created don’t automatically assign the IP to the EC2 instances or whatever other service, so, we need to enable the “auto assign IP”. In this way, we will be able to create many EC2 nodes without problems.
To do this, you need to select both subnets created and click on “Actions” and choose “Modify auto-assign IP settings”:
In the next step just check the box and save the new setting:
Ok, we’ve completed the network stuff. Now we can continue creating the ECR to upload our Docker Images for our ServerAPP.
Go to the Elastic Container Register dashboard and create a public repository with the name backend/api-service . You can use whatever name you want. If you are going to create the Docker image using your custom names, don't forget to update the following commands.
Use this url to go directly to ECR Creation Dashboard: here.
The next step is to upload our Image to ECR. So, Let’s create the Docker file to prepare the image in the root folder of our service:
## ./DockerfileFROM node:latest
WORKDIR /backend
COPY ./backend/package.json ./
RUN npm install
COPY ./backend/ ./
CMD ["npm", "start"]
In my case, I have all my server code on the backend/ folder, here I'm telling to Docker to:
- Pull an image with the latest version of NodeJS.
- Add the
backend/folder as the default workdir for our image. - Copy the package JSON to the workdir route.
- Install the dependencies and start the application.
Note: Update your package.json and add the following script:
"start": "node index.js" .
Update the Dockerfile if you have a different folder structure than me.
Uploading our service image to ECR
AWS Provides us with all the necessary commands to sync our image with the newly created ECR repository, but before this, you must authenticate in your AWS CLI. Use the following command with the credentials of the user we just created.
aws configureand follow the wizard
You’ll be able to see 4 commands that you can run in your terminal.
- The first command gets the login password for ECR and sets it directly to Docker to connect AWS with your Docker Desktop App.
- The second one is simply to build the image using the name of your repository.
- This one links the tag of the image you just created to the AWS ECR repository.
- And finally, we send the image to AWS ECR. This could take a long time while it sends all the images (~300MB) to the cloud.
We should see now the image in the ECR Repository like in the screenshot below:
Finally, we‘re done! we finished uploading the image that we need to deploy to our EKS cluster. Now we can proceed to create all the EKS and Kubernetes stuff. As we completed all the prerequisites in the IAM Users / Roles part, this should be straightforward.
Creating EKS Cluster and Nodes
Creating the EKS cluster takes a long time while Amazon sets up the infrastructure for our new cluster. To start with this, let’s go to the EKS Dashboard, here.
Before continuing, check you’re logged in with the IAM User we just created because the owner is the only user able to get data from the cluster or make changes through the AWS CLI.
If you haven’t created a cluster yet, you only have to type the name of your new cluster and go ahead. In my case, I’ll call it MyNewCluster.
In the first cluster settings, choose the latest Kubernetes version and also the role we just created in the IAM Dashboard. This role will allow EKS to handle their master and worker nodes.
Note: If you don’t see any Role to select, go back to the IAM step and create it.
Let’s move on and go to the next step in the Wizard. You can leave the other settings with the default values.
In this step, we need to assign all the network configurations of our EC2 instances as we create them. We should use the VPC we provisioned in the previous step. It will automatically take all the available subnets. In your case, you should see two of them.
The last field is about the security group. We can choose the default security group (We need to do a change in the future but not right now).
As before, we can leave all the default settings for the rest of the fields. Click on “Next” and finish the cluster creation. So, that’s all we need to do to create the cluster. It should take a long time while AWS sets up the cluster and we should see something like this:
Once the cluster has the label: “Active” we can go ahead to create the Node Group, which basically is a set of AWS EC2 Instances where the workers and pods will be located. Keep in mind each EC2 machine type has a maximum number of pods that you can create in the instance. Let’s see some examples:
Fun fact: I found this information after I created my node group and everything went wrong trying to publish my service since I was using a t3.micro instance. This instance doesn’t have enough resources to run my pods.
Creating the Node Group in EKS
To create the new Node group the cluster must have an “Active” status. Click on “Add new Node Group” and then, we’ll use the IAM Role we created in a few previous steps:
After this step, we can configure our EC2 Images in our cluster and some scalability options:
- AMI Type: here you can choose which OS or Image you want for your EC2 Machines when they are created.
- CapacityType: you can choose any of the capacity types of EC2, in this case, on-demand will bill you according to the hours in which you use the machine. See this doc to know more about EC2 Pricing.
- Instance Type: Here you can choose whatever instance you want, but I suggest you choose
t3.mediumsince it allows 17 pods for our cluster. Thet3.microinstance only allows 4 pods, and AWS by default creates 4 pods for networking and internal settings. - Disk Size: By default it sets 20GBi, but for me 10GBi are ok.
Now, we can configure our scalability rules for our Nodes. This means the behavior when EKS has to create new instances because our number of pods increased suddenly.
In this case, I just set those values to have a small application running on our EKS cluster.
- Minimum Size: When all the services have a low throughput we can decrease the number of Nodes to 1.
- Maximum Size: When all the services have a high throughput we can increase the number of Nodes to 3.
- Desired Size: Initially, the Node Group will create two nodes for our EKS Cluster.
Just click on “Next” twice and wait for the Node Group creation. It will take a long time while AWS is creating the EC2 instances and applying to the EKS master node all the settings. You can go to your EC2 dashboard and see them in the list:
Also, in the EKS dashboard, we can see the status of the Node Group creation. If everything goes fine, you should be able to see the “Active” status In the EKS Compute Dashboard and see the nodes as follows:
Connecting Kubectl in local to our EKS Cluster
Since we’ve created all the necessary resources in our AWS Account we can continue with everything regarding our local configuration and also the Deployment files for our EKS Cluster. Now we’re going to:
- Connect Kubectl with our cluster
- Create Deployments File and a Load Balancer for our APP
To connect EKS with your machine you should run the following commands:
aws sts get-caller-identitythis should respond to you with the account you have set on your aws-cli app.aws eks --region us-east-1 update-kubeconfig --name MyNewClusterThis command will configure your kubectl file to connect with AWS. You can see the file in~/.kubectl/config- Now, you should have all the settings properly configured. Let’s see how kubectl works with AWS by running:
kubectl get nodes -o wide. You should see something like:
Now, let’s create our Deployment files for our express app in node JS. Just remember we’ve already mounted the image in our ECR cluster and it’s in this step where we need it.
Creating deployment files and a Load-Balancer
Now we’re going to prepare the necessary files to give the instructions to EKS about what image it should pull and also set how many pods we need. Check out the following file:
- First, we’re declaring a “Deployment” Object type. This tells K8S how many pods should be up. It also tells Docker which image to pull from the repository. See that in line 17 we are assigning the image we uploaded before to AWS. What you should do is copy the URL of the image in ECR and paste it there.
- Just a note: You can see in different parts of the file those
---characters, this tells YAML or k8s that we’re going to create a new object below. - The next object is a service, the cluster-IP service type is in charge of making communicable the pod or your container within the node, this means other services or apps internally created can access your service.
- In the last part of the file, you can see the LoadBalancer (LB). This is a great option when we upload services to the cloud since it deploys a unique URL for accessing your app. In this case, we’re asking the LB to redirect all the internet traffic that comes from port 80 (HTTP) to port 4000 internal on the EKS Cluster.
Notes: This file can be located in the root of your project but it’s recommended to create a route like:
./infra/k8/<your-deployment-files>. Also I suggest you to update the port if your app runs in a different one.
Now is time to Deploy our service to EKS!
Now, just we’re one command closer to see our service in the Cloud powered by EKS and EC2 instances. To achieve this, we need to run the following command:
kubectl apply -f <path-to-deployment-file>
This command will sync the statements you write in the deployment file with EKS and all the pods will start to be created and initialized. If everything is ok, you should see something like this:
Now, how can we access our service? We just need to check the URL provided by AWS to my Load balancer. You can see the info running: kubetctl get services . It will respond to you with all the services including the URL in the External-IP Column.
Now, when we visit this URL we can see an error message saying the DNS is not found. Please just wait a bit longer while the URL is replicated to all the Domain name services in the world. After some minutes you can check your service is up and running like this:
Also, if you check the Dashboard of your cluster, you can see the app deployed in AWS in the workloads tab:
If you want to check how many pods are running, you can use the command: kubetctl get pods and it will let you know which are pods running for your service.
Thanks for reading! I hope this post is useful to you. Just let me know in the comments box if you release your service to EKS!
If you have questions, you can follow me on Twitter at @FlavioAandres
