Run a Private Kubernetes Cluster in a Virtual Private Cloud in AWS

23 min readApr 12, 2023

In this tutorial, we are going to create a custom VPC to host private EKS clusters. The tutorial creates the VPC in 2 ways:

The first one is using an already existing CloudFormation template offered by AWS to create the VPC.
The second one is manually creating the VPC and all of its resources to be more familiar with the details.

In another tutorial, the process will be automated by using Former2 to generate the CloudFormation script for the resources created in the second way. Then, a CloudFormation stack will be created to automatically create a private EKS cluster running in a VPC.

It is expected that the reader is familiar with the AWS Management Console and able to work with VPCs (Subnets, Route Tables, Endpoints, Network ACLs, Security Groups), CloudFormation Stacks, and IAM Roles.

The main sections are:

Introduction.
Building the VPC using CloudFormation.
Building the VPC Manually.
Create a Private EKS Cluster.
Conclusion.

Introduction

AWS Virtual Private Cloud (VPC) enables the user to create a private virtual network on the cloud with isolated resources from the other AWS Cloud. The AWS account comes with a default VPC in each region. The default VPCs have:

Default subnet.
Default Internet gateway.
Route in the main Route Table that sends traffic to the Internet gateway.

If the user created a new VPC, then it is called a nondefault VPC. By default, there is a maximum of 5 VPCs per region.

This section discusses creating a private VPC where an EKS (Elastic Kubernetes Service) cluster will be hosted. The following 2 ways are discussed:

Infrastructure-as-Code (IaC): Using the AWS IaC CloudFormation service to automatically create all the required resources (VPC, subnets, endpoints, security groups, route tables, etc).
Manually: Building the resources manually using the AWS Management Console.

Note that the benefit of manually building the VPC is to familiarize yourself with all the resources needed to be created.

Building the VPC using CloudFormation

The traditional way of creating AWS applications is to create each individual resource manually. This is useful specially for beginners to familiarize themselves with the platform and build the development environment where the developers did not yet find the optimal infrastructure for their projects.

After the development phase and deciding the resources to be created, the production phase comes where, theoretically, there are no more changes to be made.

To build the production environment, building the resources manually is not the recommended approach. Instead, use an IaC service that automates the environment creation.

AWS has an IaC service called CloudFormation to manage the AWS resources using code. Instead of building the infrastructure manually, CloudFormation builds the resources using a script (YAML or JSON) that defines the resources.

AWS has a guide to creating a VPC using existing CloudFormation templates. This guide has instructions to create a VPC with:

Public and private subnets.
Public subnets only.
Private subnets only.

We are just going to follow the instructions to create only private subnets. These steps are in the Only private subnets tab of the guide. Just open the AWS CloudFormation console and follow the steps.

The amazon-eks-fully-private-vpc.yaml YAML file is available at this link where you can download and check its content.

VPC Resource Map

Once the CloudFormation stack is created, then the VPC and all of its resources (e.g. subnets) will be created. This is the resource map of the VPC.

The VPC has the following resources:

2 Subnets
6 Endpoints
1 Route Table
2 Security Groups
1 Network ACL (default)

Remember that this VPC is created for a specific application which is hosting an EKS cluster. The resources vary for different kinds of applications.

In the following subsections, the details of each resource are listed in a table. This helps later when building them manually. Some columns (e.g. Name and Owner ID) are removed from some tables.

Subnets

The stack has 2 subnets.

| Name                     | Subnet ID                | State     | VPC                                   | IPv4 CIDR       | IPv6 CIDR | Available IPv4 addresses | Availability Zone | Availability Zone ID | Network border group | Route table           | Network ACL           | Default subnet | Auto-assign public IPv4 address | Auto-assign customer-owned IPv4 address | Customer-owned IPv4 pool | Auto-assign IPv6 address |
| :----------------------- | :----------------------- | :-------- | :------------------------------------ | :-------------- | :-------- | :----------------------- | :---------------- | :------------------- | :------------------- | :-------------------- | :-------------------- | :------------- | :------------------------------ | :-------------------------------------- | :----------------------- | :----------------------- |
| rc-stack-PrivateSubnet01 | subnet-0a2d20126829d5028 | Available | vpc-0c5d514fed963989e \| rc-stack-VPC | 192.168.0.0/18  | –         | 16373                    | ca-central-1a     | cac1-az1             | ca-central-1         | rtb-022070e1699b11705 | acl-0d57ff06d2c2318e2 | No             | No                              | No                                      | -                        | No                       |
| rc-stack-PrivateSubnet02 | subnet-073410e20eab6a965 | Available | vpc-0c5d514fed963989e \| rc-stack-VPC | 192.168.64.0/18 | –         | 16373                    | ca-central-1b     | cac1-az2             | ca-central-1         | rtb-022070e1699b11705 | acl-0d57ff06d2c2318e2 | No             | No                              | No                                      | -                        | No                       |

Endpoints

6 endpoints are created for these 6 services:

com.amazonaws.{REGION}.sts
com.amazonaws.{REGION}.logs
com.amazonaws.{REGION}.ecr.api
com.amazonaws.{REGION}.ecr.dkr
com.amazonaws.{REGION}.ec2
com.amazonaws.{REGION}.s3

Remember to change the region name {REGION} according to the region used in the VPC.

| VPC endpoint ID        | VPC ID                                | Service name                       | Endpoint type | Status    | Network interfaces   | Subnets   |
| :--------------------- | :------------------------------------ | :--------------------------------- | :------------ | :-------- | :------------------- | :-------- |
| vpce-030ce3f319c3ddb7d | vpc-0c5d514fed963989e \| rc-stack-VPC | com.amazonaws.ca-central-1.sts     | Interface     | Available | 2 network interfaces | 2 subnets |
| vpce-0da5622e3c89a14a1 | vpc-0c5d514fed963989e \| rc-stack-VPC | com.amazonaws.ca-central-1.logs    | Interface     | Available | 2 network interfaces | 2 subnets |
| vpce-067d7bd7c4ef926bb | vpc-0c5d514fed963989e \| rc-stack-VPC | com.amazonaws.ca-central-1.ecr.api | Interface     | Available | 2 network interfaces | 2 subnets |
| vpce-05fbbc273cb45bd28 | vpc-0c5d514fed963989e \| rc-stack-VPC | com.amazonaws.ca-central-1.ecr.dkr | Interface     | Available | 2 network interfaces | 2 subnets |
| vpce-05793738781ed83f0 | vpc-0c5d514fed963989e \| rc-stack-VPC | com.amazonaws.ca-central-1.ec2     | Interface     | Available | 2 network interfaces | 2 subnets |
| vpce-0a728bc8e5bc8bd21 | vpc-0c5d514fed963989e \| rc-stack-VPC | com.amazonaws.ca-central-1.s3      | Gateway       |           |                      |           |

Route Table

There is only 1 Route Table created by the stack. Note that the Main Route Table with ID rtb-06590ff533a888b5d is automatically created when the VPC is created. It controls the routes to/from the VPC Subnets not already explicitly associated with any Route Table.

Because the 2 subnets are already associated with the custom Route Table rtb-022070e1699b11705, then the Main Route Table has no subnets associated.

| Route table ID        | Explicit subnet associations | Edge associations | Main | VPC                                   |
| :-------------------- | :--------------------------- | :---------------- | :--- | :------------------------------------ |
| rtb-06590ff533a888b5d | –                            | –                 | Yes  | vpc-0c5d514fed963989e \| rc-stack-VPC |
| rtb-022070e1699b11705 | 2 subnets                    | –                 | No   | vpc-0c5d514fed963989e \| rc-stack-VPC |

These are the routes in the custom Route Table rtb-022070e1699b11705. The first route is automatically added to the route table when the com.amazonaws.ca-central-1.s3 gateway endpoint is created.

| Destination    | Target                 | Status | Propagated |
| :------------- | :--------------------- | :----- | :--------- |
| pl-7da54014    | vpce-0a728bc8e5bc8bd21 | Active | No         |
| 192.168.0.0/16 | local                  | Active | No         |

Security Groups

Only 2 Security Groups are created. There is a third group which is the default SecurityGroup.

If you don’t specify a security group when you launch an instance, the instance is automatically associated with the default security group for the VPC. A default security group is named default.

| Security group ID    | Security group name                             | VPC ID                | Description                                           | Inbound rules count  | Outbound rules count |
| :------------------- | :---------------------------------------------- | :-------------------- | :---------------------------------------------------- | :------------------- | -------------------- |
| sg-0161ad0f4a49812a2 | rc-stack-ControlPlaneSecurityGroup-K5UEDVQCQCSV | vpc-0c5d514fed963989e | Cluster communication with worker nodes               | 0 Permission entries | 1 Permission entry   |
| sg-0ab375769c66ce0b3 | default                                         | vpc-0c5d514fed963989e | default VPC security group                            | 1 Permission entry   | 1 Permission entry   |
| sg-0fd6b1096799997cf | rc-stack-EndpointSecurityGroup-10ZH7KUW442W4    | vpc-0c5d514fed963989e | Security group to govern who can access the endpoints | 1 Permission entry   | 1 Permission entry   |

The rc-stack-ControlPlaneSecurityGroup security group does not have any inbound rules and only have the default outbound rule.

For the rc-stack-EndpointSecurityGroup security group, it is associated with the endpoints. It has this inbound rule.

| Security group rule ID | IP version | Type  | Protocol | Port range | Source         | Description |
| :--------------------- | :--------- | :---- | :------- | :--------- | :------------- | ----------- |
| sgr-0518dbdce783b0fba  | IPv4       | HTTPS | TCP      | 443        | 192.168.0.0/16 | –           |

The outbound rule in the rc-stack-EndpointSecurityGroup security group.

| Security group rule ID | IP version | Type        | Protocol | Port range | Destination | Description |
| :--------------------- | :--------- | :---------- | :------- | :--------- | :---------- | ----------- |
| sgr-0af87ecd6b6878ad9  | IPv4       | All traffic | All      | All        | 0.0.0.0/0   | –           |

Network ACL

The stack did not create a custom Network Access Control List (ACL). Instead, it just used the default ACL with the ID acl-0d57ff06d2c2318e2. But the ACL is updated to include another inbound rule and another outbound rule to allow traffic to go into/from the subnets.

Inbound Rules

The stack created a new inbound rule with the number 100 to allow the incoming traffic to the subnet.

| Rule number | Type        | Protocol | Port range | Source    | Allow/Deny |
| :---------- | :---------- | :------- | :--------- | :-------- | :--------- |
| 100         | All traffic | All      | All        | 0.0.0.0/0 | Allow      |
| *           | All traffic | All      | All        | 0.0.0.0/0 | Deny       |

Outbound Roles

The stack created a new outbound rule with the number 100 to allow the outgoing traffic from the subnet.

| Rule number | Type        | Protocol | Port range | Destination | Allow/Deny |
| :---------- | :---------- | :------- | :--------- | :---------- | :--------- |
| 100         | All traffic | All      | All        | 0.0.0.0/0   | Allow      |
| *           | All traffic | All      | All        | 0.0.0.0/0   | Deny       |

Now, we know all the resources created in the CloudFormation stack. The next section manually creates a VPC manually to be more familiar with every step.

Building the VPC Manually

In this section, we are going to create a new VPC manually using the AWS Management Console. After the VPC is created, the other resources will be created and associated with the VPC.

These are the resources that we should create/modify:

1. VPC

2. Subnets

3. Endpoints

4. Security Groups

5. Route Table + Associating the Subnets with it.

6. ACL + Associating the Subnets with it.

Our target is to make sure that the VPC and its resources are matching those created by the CloudFormation template.

Make sure that you are selecting the proper region before starting. In this example, the ca-central-1 region is selected in this example.

VPC

This section creates the virtual private cloud (VPC) to host a private EKS cluster.

To get started, go to the VPC AWS service. Then click Create VPC. Make sure the proper region is selected.

There are 2 options when creating a VPC:

VPC only: Create only the VPC without any additional resources. It just creates a default security group, a default ACL, and a default route table.
VPC and more: Create other resources with the VPC. The resources include subnets, gateways, route tables, endpoints, etc.

Let’s go with the first option to create the VPC only. The only 2 configuration parameters to edit are:

The VPC name: rc-vpc.
The IPv4 CIDR block associated with the VPC: 10.0.0.0/16.

After the VPC is created, we have to edit its settings to set the enableDnsSupport and enableDnsHostnames VPC attributes to true. In other words, make sure the following 2 checkboxes are checked:

Enable DNS resolution
Enable DNS hostnames

To do this, follow these steps:

Go to the VPC.
From the Actions menu, click Edit VPC settings.
In the DNS settings section, make sure that the 2 checkboxes mentioned above are checked.
Click Save.

These are the details of the VPC.

At the moment, this is the resource map of the VPC. There is only a default Route Table.

Note that the VPC comes with a default Route Table and a default Network ACL. If you will not create a custom Route Table and Network ACL, then you can associate the subnets with them. But we are going to create custom non-default resources.

Before creating the VPC subnets, let’s create the Route Table and the Network ACL that will be associated with the VPC subnets.

Subnet Route Table

At the moment, we are using the default (main) route table rtb-05dabe130ec8df650. The reason why we want to create a custom Route Table is that we cannot reference the default resources in the CloudFormation code. So, it is recommended to create custom resources unless you are confident that you will not be using CloudFormation for automating the process.

Sometimes we want to add or remove a route from the default Route Table. In the CloudFormation code, we cannot reference the default Route Table. Thus, we are unable to add/remove routes. So, we will create a custom Route Table which we can reference in the CloudFormation code. This Route Table will be associated with:

The subnets.
The com.amazonaws.ca-central-1.s3 Endpoint.

Follow these steps to create a new Route Table:

Go to the VPC service. Make sure you select the proper region.
Click Route tables from the left bar under the Virtual private cloud section.
Click Create route table.
Enter the Route Table name (e.g. rc-vpc-route-table).
Select the VPC (e.g. rc-vpc).
Click Create route table.

Let’s assume that the ID of this new Route Table is rtb-0dc8d1d07266ed127. Later, we will associate this Route Table with the subnets.

Next is to create the network access control list (ACL).

Subnet Network ACL

Similar to the Route Table, we have to create a custom network ACL instead of using the default. This ACL will be associated with the subnets. This is for the same reason which is that we cannot reference the default network ACL in the CloudFormation script and so we cannot add/remove rules.

Follow these steps to create a new network ACL:

Go to the VPC service. Make sure you select the proper region.
Click Network ACLs from the left bar under the Security section.
Click Create network ACL.
Enter the ACL name (e.g. rc-vpc-subnet-acl).
Select the VPC (e.g. rc-vpc).
Click Create network ACL.

The ID of this new ACL is acl-0885f82f805f6c61f.

After the ACL is created, then there is only a single inbound Deny rule and a single outbound Deny rule. This means no traffic will enter or exit the subnets associated with this ACL.

| Rule number | Type        | Protocol | Port range | Source    | Allow/Deny |
| :---------- | :---------- | :------- | :--------- | :-------- | :--------- |
| *           | All traffic | All      | All        | 0.0.0.0/0 | Deny       |

We have to add a new Allow rule for both inbound and outbound packets.

To add the Allow inbound rule, follow these steps:

Go to the ACL acl-0885f82f805f6c61f.
In the Inbound rules tab, click Edit inbound rules.
Click Add new rule.
In the Rule number, enter a number like 100.
For the Type, select All traffic.
Make sure that the Source is set to 0.0.0.0/0.
Make sure that the rule is Allow.

These are the inbound rules after adding the new Allow rule.

| Rule number | Type        | Protocol | Port range | Source    | Allow/Deny |
| :---------- | :---------- | :------- | :--------- | :-------- | :--------- |
| 100         | All traffic | All      | All        | 0.0.0.0/0 | Allow      |
| *           | All traffic | All      | All        | 0.0.0.0/0 | Deny       |

For the Allow outbound rule, follow these steps:

1. Go to the ACL acl-0885f82f805f6c61f.

2. In the Outbound rules tab, click Edit Outbound rules.

3. Click Add new rule.

4. In the Rule number, enter a number like 100.

5. For the Type, select All traffic.

6. Make sure that the Source is set to 0.0.0.0/0.

7. Make sure that the rule is Allow.

The outbound rules look identical to the table above.

Once we created both the Route Table and the Network ACL, then we are ready to create the VPC Subnets.

Subnet

To create the VPC subnets, follow these steps:

Go to the VPC service.
From the left bar, select click on Subnets from the Virtual private cloud group.
Click Create subnet.
On the subnet creation page, we can create 1 or more subnets by clicking on the Add new subnet button to add an extra subnet. Let’s create 2 subnets.

Remember that the IPv4 CIDR block of the VPC is 10.0.0.0/16.

For the first subnet:

Select the VPC rc-vpc.
In the Subnet settings section, specify the following:

1. Subnet name: rc-vpc-subnet-01

2. Availability Zone: ca-central-1a

3. IPv4 CIDR block: 10.0.0.0/18.

4. Add this new tag:

Tag Key: kubernetes.io/role/internal-elb
Tag Value: 1.

Click on the Add new subnet button to add the second subnet.

Select the VPC rc-vpc.
In the Subnet settings section, specify the following:
- Subnet name: rc-vpc-subnet-02- Availability Zone: ca-central-1b- IPv4 CIDR block: 10.0.64.0/18.
- Add this tag to expose the subnets in case you will use AWS Load Balancer Controller:
— Tag Key : kubernetes.io/role/internal-elb — Tag Value: 1.

Finally, click on the Create subnet button to create the 2 subnets. Here are the details of the 2 subnets. Once the subnet is created, then it uses the default Route Table and the default Network ACL. We have to explicitly associate the subnets with the custom Route Table and the custom Network ACL.

| Name             | Subnet ID                | State     | VPC                             | IPv4 CIDR    | IPv6 CIDR | Available IPv4 addresses | Availability Zone | Availability Zone ID | Network border group | Route table           | Network ACL           | Default subnet | Auto-assign public IPv4 address | Auto-assign customer-owned IPv4 address | Customer-owned IPv4 pool | Auto-assign IPv6 address |
| :--------------- | :----------------------- | :-------- | :------------------------------ | :----------- | :-------- | :----------------------- | :---------------- | :------------------- | :------------------- | :-------------------- | :-------------------- | :------------- | :------------------------------ | :-------------------------------------- | :----------------------- | :----------------------- |
| rc-vpc-subnet-01 | subnet-023ee7285b69819e9 | Available | vpc-0726a9456a97a2214 \| rc-vpc | 10.0.0.0/18  | –         | 16379                    | ca-central-1a     | cac1-az1             | ca-central-1         | rtb-05dabe130ec8df650 | acl-023405ff7873aa3c4 |                |                                 |                                         |                          |                          |
| rc-vpc-subnet-02 | subnet-081a1b1efbd5a0ded | Available | vpc-0726a9456a97a2214 \| rc-vpc | 10.0.64.0/18 | –         | 16379                    | ca-central-1b     | cac1-az2             | ca-central-1         | rtb-05dabe130ec8df650 | acl-023405ff7873aa3c4 | No             | No                              | No                                      | -                        | No                       |

This is the VPC resource map after creating the 2 subnets. Because the subnets are not associated with a Route Table, then they are automatically associated with the default Route Table.

Associate the Route Table with the Subnets

The subnets are associated with the default Route Table. To associate the subnets with the custom Route Table rtb-0dc8d1d07266ed127, then follow these steps:

Go to the subnet.
Click on the Actions menu.
Click on the Edit route table association option.
Select the route table rtb-0dc8d1d07266ed127.
Click Save.

Repeat the steps for each subnet.

This is how the subnets look like after changing the Route Table.

| Name             | Subnet ID                | State     | VPC                             | IPv4 CIDR    | IPv6 CIDR | Available IPv4 addresses | Availability Zone | Availability Zone ID | Network border group | Route table                                 | Network ACL           | Default subnet | Auto-assign public IPv4 address | Auto-assign customer-owned IPv4 address | Customer-owned IPv4 pool | Auto-assign IPv6 address |
| :--------------- | :----------------------- | :-------- | :------------------------------ | :----------- | :-------- | :----------------------- | :---------------- | :------------------- | :------------------- | :------------------------------------------ | :-------------------- | :------------- | :------------------------------ | :-------------------------------------- | :----------------------- | :----------------------- |
| rc-vpc-subnet-01 | subnet-023ee7285b69819e9 | Available | vpc-0726a9456a97a2214 \| rc-vpc | 10.0.0.0/18  | –         | 16379                    | ca-central-1a     | cac1-az1             | ca-central-1         | rtb-0dc8d1d07266ed127 \| rc-vpc-route-table | acl-023405ff7873aa3c4 |                |                                 |                                         |                          |                          |
| rc-vpc-subnet-02 | subnet-081a1b1efbd5a0ded | Available | vpc-0726a9456a97a2214 \| rc-vpc | 10.0.64.0/18 | –         | 16379                    | ca-central-1b     | cac1-az2             | ca-central-1         | rtb-0dc8d1d07266ed127 \| rc-vpc-route-table | acl-023405ff7873aa3c4 | No             | No                              | No                                      | -                        | No                       |

This is the new VPC resource map after associating the 2 subnets with the new Route Table. The default table is no longer used.

Associate the Network ACL with the Subnets

Similar to associating the subnets with the custom Route Table, we will associate them with the custom Network ACL.

To associate the subnets with the custom Network ACL acl-0885f82f805f6c61f, then follow these steps:

Go to the subnet.
Click on the Actions menu.
Click on the Edit network ACL association option.
Select the Network ACL acl-0885f82f805f6c61f.
Click Save.

Repeat the steps for each subnet.

This is how the subnets look like after changing the Network ACL.

| Name             | Subnet ID                | State     | VPC                             | IPv4 CIDR    | IPv6 CIDR | Available IPv4 addresses | Availability Zone | Availability Zone ID | Network border group | Route table                                 | Network ACL                                | Default subnet | Auto-assign public IPv4 address | Auto-assign customer-owned IPv4 address | Customer-owned IPv4 pool | Auto-assign IPv6 address |
| :--------------- | :----------------------- | :-------- | :------------------------------ | :----------- | :-------- | :----------------------- | :---------------- | :------------------- | :------------------- | :------------------------------------------ | :----------------------------------------- | :------------- | :------------------------------ | :-------------------------------------- | :----------------------- | :----------------------- |
| rc-vpc-subnet-01 | subnet-023ee7285b69819e9 | Available | vpc-0726a9456a97a2214 \| rc-vpc | 10.0.0.0/18  | –         | 16379                    | ca-central-1a     | cac1-az1             | ca-central-1         | rtb-0dc8d1d07266ed127 \| rc-vpc-route-table | acl-0885f82f805f6c61f rc-vpc-subnet-acl    |                |                                 |                                         |                          |                          |
| rc-vpc-subnet-02 | subnet-081a1b1efbd5a0ded | Available | vpc-0726a9456a97a2214 \| rc-vpc | 10.0.64.0/18 | –         | 16379                    | ca-central-1b     | cac1-az2             | ca-central-1         | rtb-0dc8d1d07266ed127 \| rc-vpc-route-table | acl-0885f82f805f6c61f \| rc-vpc-subnet-acl | No             | No                              | No                                      | -                        | No                       |

Endpoint Security Group

For the same reason of creating the custom Route Table and custom Network ACL, we have to create a custom Security Group so that we can reference and edit it from the CloudFormation code.

Follow these steps to create a new custom Security Group for the endpoints:

Go to the VPC service. Make sure you select the proper region.
Click Security groups from the left bar under the Security section.
Click Create security group.
Enter the Security Group name (e.g. rc-vpc-security-group).
Select the VPC (e.g. rc-vpc).
In the Inbound rules section, add a new rule by following these steps:
- Click Add rule.
- For Type, select HTTPS.
- For Source, enter the VPC CIDR block 10.0.0.0/16.
Click Create security group.

This is the new Security Group sg-0c027a84eac42cb26. This is how the Inbound Rule looks like in the Inbound rules tab of the Security Group.

| Security group rule ID | IP version | Type  | Protocol | Port range | Source      |
| :--------------------- | :--------- | :---- | :------- | :--------- | :---------- |
| sgr-0488a0352f88169f5  | IPv4       | HTTPS | TCP      | 443        | 10.0.0.0/16 |

If you already created the custom Security Group without adding the Inbound Rule, then we can make add the Inbound Rule by following these steps:

Identify the security group to be associated with the endpoints. If the Endpoints are already associated, then you can visit any Interface endpoint. Down on the page, go to the Security Groups tab. There we will find the security group associated.
Go to the security group. Down on the page, go to the Inbound Rules tab. Click Edit inbound rules.
Click the Add rule button and specify the details of the rule:
- Type: HTTPS.
- Protocol: TCP.
- Port range: 443.
- Source: 10.0.0.0/16.

Now, we have the VPC and its subnets ready. Let’s create the endpoints to make sure the VPC subnets have connectivity with the other services needed in our application which is running worker nodes in a private EKS cluster.

Endpoint

We need to create the following 6 endpoints to have connectivity between the subnets and other AWS services needed to run the EKS clusters. Remember to change the region name to the VPC region.

com.amazonaws.ca-central-1.sts
com.amazonaws.ca-central-1.logs
com.amazonaws.ca-central-1.ecr.api
com.amazonaws.ca-central-1.ecr.dkr
com.amazonaws.ca-central-1.ec2
com.amazonaws.ca-central-1.s3

Note that the endpoint type for all the target endpoints is Interface except for com.amazonaws.ca-central-1.s3 which has the type Gateway.

To create the VPC subnets, follow these steps:

Go to the VPC service.
From the left bar, select click on Endpoints from the Virtual private cloud group. Then click Create endpoint. For each endpoint, do the following:
Enter the endpoint name. For example, the name for the sts endpoint is rc-vpc-endpoint-sts.
Select the AWS services from the Service category.
From the Services section, search for the target service then select it.
Select the rc-vpc VPC.
Select the subnets associated with the endpoint. Let’s select all the 2 subnets.
Set IP address type to IPv4.
Select the custom Security Group sg-0c027a84eac42cb26.
Set Policy to Full access.
Click Create endpoint.

Repeat the above steps until creating the other 5 endpoints. For the com.amazonaws.ca-central-1.s3 service, you will select the custom Route Table rtb-0dc8d1d07266ed127 instead of the custom Security Group.

These are the 6 endpoints created.

| Name                    | VPC endpoint ID        | VPC ID                          | Service name                       | Endpoint type | Status    | Network interfaces   | Subnets   | Route tables                                |
| :---------------------- | :--------------------- | :------------------------------ | :--------------------------------- | :------------ | :-------- | :------------------- | :-------- | :------------------------------------------ |
| rc-vpc-endpoint-sts     | vpce-0e9259dc2256932ca | vpc-0726a9456a97a2214 \| rc-vpc | com.amazonaws.ca-central-1.sts     | Interface     | Available | 2 network interfaces | 2 subnets | –                                           |
| rc-vpc-endpoint-logs    | vpce-0fbbb0ec919104385 | vpc-0726a9456a97a2214 \| rc-vpc | com.amazonaws.ca-central-1.logs    | Interface     | Available | 2 network interfaces | 2 subnets | –                                           |
| rc-vpc-endpoint-ecr-api | vpce-0f661c4cd27d370e2 | vpc-0726a9456a97a2214 \| rc-vpc | com.amazonaws.ca-central-1.ecr.api | Interface     | Available | 2 network interfaces | 2 subnets | –                                           |
| rc-vpc-endpoint-ecr-dkr | vpce-015e0186049c6ccd8 | vpc-0726a9456a97a2214 \| rc-vpc | com.amazonaws.ca-central-1.ecr.dkr | Interface     | Available | 2 network interfaces | 2 subnets | –                                           |
| rc-vpc-endpoint-ec2     | vpce-01f0b9a9333f1ece7 | vpc-0726a9456a97a2214 \| rc-vpc | com.amazonaws.ca-central-1.ec2     | Interface     | Available | 2 network interfaces | 2 subnets | –                                           |
| rc-vpc-endpoint-s3      | vpce-0c10a5c87794c609f | vpc-0726a9456a97a2214 \| rc-vpc | com.amazonaws.ca-central-1.s3      | Gateway       | Available | –                    | –         | rtb-0dc8d1d07266ed127 \| rc-vpc-route-table |

If the Endpoint is associated with a wrong Security Group, then follow these steps to change the Security Group associations:

Go to the Endpoint.
Click on the Actions menu.
Click on the Manage security groups option.
Select the Security Groups to be associated with the endpoint.
Click Modify security groups.

To change the Endpoint’s subnet association after the Endpoint is created, then follow these steps:

Go to the Endpoint.
Click on the Actions menu.
Click on the Manage subnets option.
Select the Subnets to be associated with the endpoint.
Click Modify endpoints.

This is the new VPC resource map after creating the endpoints. It might take some minutes until the subnets get connected with the S3 endpoint.

Now, the VPC is ready. Just create a new EKS cluster and associate it with the rc-vpc VPC.

Create a Private EKS Cluster

This section creates a private EKS cluster inside the VPC just created. Then, a worker Node Group is created to run some worker nodes. Two IAM roles are created which are needed by the cluster and the nodes.

Create the IAM Roles

Both the EKS Cluster and the Worker NodeGroup need to be assigned IAM roles so that they have permission to manage the resources they need (e.g. creating EC2 instances). The next 2 subsections create 2 IAM roles. The first one is for the EKS cluster. The second one is for the Worker NodeGroup inside the cluster.

IAM Role for the EKS Cluster

Just follow these steps to create the IAM role for the EKS cluster:

Go to the IAM service.
Go to the Roles page.
Click the Create role button.
In the Trusted entity section, select AWS service.
In the Use case section, choose EC2. Its subtitle is Allows EC2 instances to call AWS services on your behalf.
Click the Next button. Then search and select these 3 policies.
AmazonEKSClusterPolicy: This policy grants Kubernetes the required permissions to manage resources.
AmazonEKS_CNI_Policy: This policy provides the Amazon VPC CNI Plugin the permissions it requires to modify the IP address configuration on your EKS worker nodes.
Click the Next button. Enter the role name (e.g. rc-cluster-role) and click the Create role button.

IAM Role for the Worker NodeGroup

Repeat the previous steps except for step 6 as we need the following 3 policies to be assigned to the role:

AmazonEKSWorkerNodePolicy: This policy allows Amazon EKS worker nodes to connect to Amazon EKS Clusters.
AmazonEC2ContainerRegistryReadOnly: Provides read-only access to Amazon EC2 Container Registry repositories.
AmazonEKS_CNI_Policy: This policy provides the Amazon VPC CNI Plugin the permissions it requires to modify the IP address configuration on your EKS worker nodes.

Let’s give this role the name rc-worker-role. This role gives the worker nodes permissions to connect to the EKS cluster.

Create the Cluster

Follow these steps to create the EKS cluster:

Go to the Elastic Kubernetes Service (EKS) in the AWS Management Console.
Click Add clusters and select Create.
Enter the cluster name, select the Kubernetes version, and select the IAM role which is rc-cluster-role.
Click Next to specify the networking configuration. Select the VPC created previously. Keep all the subnets in the VPC selected. For the security group, we can use the default group. For Cluster endpoint access, set it to Private so that the cluster is accessible only within the VPC.
There is nothing more to change. Just click Next to skip the other steps and finally create Create to create the EKS cluster.

Create the Worker NodeGroup

On the home page of the cluster, there is a tab named Compute. There we can manage the computing/worker nodes or node groups associated with the cluster. It is empty now.

We can add a worker node group to the cluster by going down and clicking the Add node group button in the Node groups section.

We have to select an IAM role for the computing worker node group. Just select the role rc-worker-role created previously.

On the next page, there are 3 sections to configure the node group:

Node group compute configuration: This is where the EC2 instance is configured.
Node group scaling configuration: To configure the number of worker nodes in the group and how they scale.
Node group update configuration: The maximum number/percentage of unavailable worker nodes to be tolerated.

Click the Next button after configuring the node group according to your preferences. Then specify the network configuration for the node group. This includes selecting the subnets in which the nodes of this node group will be created.

Then, click the Next button and click Create. It takes some minutes to get the worker node group created.

In the Compute tab of the EKS cluster, the nodes will be listed. Their status is Ready which means they are working properly.

Conclusion

The tutorial discussed creating a VPC for a private EKS cluster. The resources are created in 2 ways:

CloudFormation template offered by AWS.
Manually by creating each resource individually.

In another tutorial, the Former2 tool will be used to create a CloudFormation template from existing resources just to automate the process of creating custom resources in the future.