AWS : Write Infrastructure as Code using Cloudformation

Create a basic infrastucture in AWS using Cloudformation and deploy a highly available simple website.

In this post, we are going to see step-by-step, how we can model and provision an environment in AWS, by defining all the needed infrastructure components using Cloudformation. Our goal is to be able to host a highly available website, which will be served from EC2 instances located in private subnets (for better security).

In the described example, in order to write our .template file, we will be using the YAML format . Apart from the required Resources section, we will have a look at some of the optional ones, such as Parameters, Conditions, Mappings and Metadata.

After we finish writing our script and creating our stack, our infrastructure will look like this:

Design of our infrastructure and resources

You can view or download the complete script from my Github repo.

Let’s start by defining the AWSTemplateFormatVersion and a short Description:

Parameters

Next, we are going to define a few Parameters, in order to make our script more generic. The AWS users that will run the template, will have to provide values to those parameters, after they upload the .template file.

We start, by giving the user the option to define a range of IP addresses from which they can access the bastion hosts we will later create. For example, when I run the template, I’m passing my own IP address (172.58.43.122/32).

We also define a parameter for the name of the key-pair file that the user will use to connect to all EC2 instances.

We then try to make the creation of the VPC and Subnets a bit more dynamic, by allowing the user to define the VCP CIDR block they want, as well as the number of subnet bits for every created subnet’s CIDR. We will see how these are going to be used when we’ll later create the corresponding components, making use of the Fn::Cidr function.

You can use a more restrictive regex expression at the AllowedPattern key

At the end of the Parameters section, we’ll give the option to the user to decide whether they want to create the private resources (EC2 instances, NAT Gateways, Elastic IPs, etc), in case someone wants to remain on the free-tier.

Please note here that enabling the creation of private resources, will incur a very small cost, as AWS charges $0.045/hour as the Price per NAT gateway.

At the end of the section, we define a list of EC2 instance types that can be used when creating both the bastion hosts as well as the website instances. We have chosen t2.micro as the default value, as we don’t want to get charged for EC2 resource usage.

In our example we are going to create the private resources

Conditions

Conditions provides us with the flexibility to control the creation of certain resources in our stack. In our example, we’ll just define a condition, that will take into consideration the value that the user will pass to the PrivateResourcesCreation parameter declared above.

The condition will be used as a check when deciding on whether certain parts of our infrastructure should be built or not.

Mappings

What follows is the Mappings section, at which we have all AMI ids for the Amazon Linux 2 image, and based on the region that the stack will be created at, the script will choose the appropriate AMI for the EC2 instances creation.

AMI ids might be changed from time to time, so please crosscheck the correctness of the data

Resources

This will be the biggest section of our template file (at least in the specific example), as we are going to define each resource needed for our infrastructure, as well as the associations between some of those.

We’ll try to write the code based on the order that we want our resources to be created, but that doesn’t play any role in the actual event execution sequence. As a helper tool, we can use the DependsOn attribute, in order to ensure that the creation of a specific resource follows another.

VPC

We start by the VPC, referencing the VpcCidrBlock defined in the Parameters section. We also prefix the Name tag with the stack name given by the user.

Next, we define the Internet Gateway and attach it to the VPC.

Subnets

In order to have high availability for our resources (both for bastion hosts & for our website app), we are going to set up resources in 2 different Availability Zones.

We’ll therefore create subnets in both AZs. In each one we’ll build one public subnet…

We create 1 public subnet in each AZ

…and if the CreatePrivateResources condition (defined above) is true, we’ll also create 1 private subnet.

We use the Condition to determine whether we’ll build private subnets as well

At this point, let’s have a closer look on how:

• the Availability Zone that the subnet will reside is determined
• each subnet’s CIDR block gets assigned

For the first one, things are pretty simple, as we use Cloudformaation’s Fn::GetAZs which returns an array with all the AZs for the specific region that we are running the template.

Regarding the subnet’s CIDR assignment, we are using the Fn:Cidr function, which gives us the option to provide:

• an initial CIDR block (in our case that of the VPC)
• the number CIDRs to generate (we’ll calculate one for every subnet)
• the number of subnet bits for the CIDR (e.g. a value of “8” creates a /24 mask)

If we have decided to hardcode the subnet CIDR values to the script, we would have to calculate the range of IPs (possibly with the help of a tool like ipaddressguide.com) and assign them to our subnets. More importantly though, in case we needed to run the same script again (let’s say in order to create the same stack, but for a testing environment) and we wanted a different mask value for our subnets, we would have to find all subnets declarations, re-calculate the IP ranges and manually make all changes to the script.

By giving the option to the user to decide on the VPC CIDR block range, and by using the Fn:Cidr function, we can dynamically produce the appropriate CIDR ranges for our subnets, based on the mask size provided by the user during passing the Parameters values, and also taking into consideration the CreatePrivateResources flag.

So, let’s assume that we have a VPC with a CIDR block value of 10.10.0.0/20, therefore a range of IPs from 10.10.0.0 → 10.10.15.255.

Fn::Cidr:
    - 10.10.0.0/20
    - 2
    - 8
(or)
!Cidr: [ 10.10.0.0/20, 2, 8 ]

will automatically create for us the next 2 subnets:

• Subnet A: 10.10.0.0/24 (IP range: 10.10.0.0 → 10.10.0.255)
• Subnet B: 10.10.1.0/24 (IP range: 10.10.1.0 → 10.10.1.255)

while

!Cidr: [ 10.10.0.0/20, 4, 6 ]

will automatically create for us the next 4 subnets:

• Subnet A: 10.10.0.0/26 (IP range: 10.10.0.0 → 10.10.0.63)
• Subnet B: 10.10.0.64/26 (IP range: 10.10.0.64 → 10.10.0.127)
• Subnet C: 10.10.0.128/26 (IP range: 10.10.0.128 → 10.10.0.191)
• Subnet D: 10.10.0.192/26 (IP range: 10.10.0.192 → 10.10.0.255)

We could make things even more dynamic, by giving to the user the option to set the number of subnets they want to create (let’s say a Parameter attribute named NumberOfSubnets) and have the CidrBlock key looking like this:

CidrBlock: !Select
    - 2
    - Fn::Cidr:
        - !Ref VpcCidrBlock
        - !Ref NumberOfSubnets
        - !Ref IPv4MaskSize

NAT Gateways & Elastic IPs

In order for the EC2 instances in our private subnets to have access to the internet, we have to create 2 NAT Gateways (1 in each AZ) and the Elastic IPs that will be allocated to them.

Please note here, that our NAT Gateways will have to reside on our public subnets.

NAT Gateways reside on the public subnets

Route Tables

We move on by creating Route Tables, adding the appropriate routes to them and finally associate our subnets to each table.

Initially, we create a public route table and we add a route to the Internet Gateway in case a resource wants to access the internet (0.0.0.0/0).

We then create 2 private route tables, as we have 2 NAT Gateways and we cannot have a single route (in our case to 0.0.0.0/0) in a route table pointing to more than one NAT Gateways.

Finally, we associate our 2 public subnets to our public route table (so that all of their resources access the internet though the IGW) and each of our 2 private subnets to a private route table (so that their resources access the internet though the NAT GW).

Network Control Access Lists (NACLs)

Our next step is to create 2 NACLs for our infrastructure. We are going to keep things simple and have both NACL’s Engress & Ingress rules to ALLOW ALL, but we’ll have the definitions there in case we want to change things in the future.

Our public NACL with the respective entries for Inbound and Outbound traffic

Define a private NACL for future use

We then associate our public subnets to our public NACL and the private subnets to the private one.

Security Groups

The last thing we need to define before we move on with the creation of our EC2 instances, is the security groups that those instances will be using.

We will define one security group for the EC2 instances that will deployed in our public subnets, and another one for the ones deployed at the private ones.

Let’s see how we will configure our public security group. We need to be able to SSH into our bastion hosts from the SSHLocation (declared in Parameters), ping them, and we also need to allow HTTP access for the Application Load Balancer we’ll create later on. From within the servers, there will be no limitations as they will be able to use all protocols and ports when reaching the internet.

The setup for the private security group will be quite similar, with the only deifference being that CIDR IP range that will accept traffic from, will be the CIDR of the VPC.

Private Security Group access inbound traffic only from within VPC

EC2 Instances : Bastion Hosts

Since we’re building a high availability infrastructure, we’ll create one EC2 instance that will act as a bastion host, in each AZ. The AMI that will be used for those instances to be created, will be retrieved from the predefined AWSRegionLinux2AMI list in the Mappings section. Finally, its type will be selected as a value to the InstanceType Parameters attribute.

EC2 Instances : Website

We have finally reached the point where we will declare the definition for our EC2 website instances. Once again, we’ll create one instance in every AZ.

For our website we will install an Apache HTTP server, which will host a simple web page, that will display a “Hello” message mentioning the hostname of the server and the availability zone that the EC2 is hosted at.

In order to set everything up along with the creation of our EC2 instances, we’ll provide all the necessary information in the UserData section of our definition.

At this point, we have a complete script that we could run and allow us to SSH into our servers, and check that everything have been set up they way they should. For example, we could SSH into our bastion hosts and make a cURL request to any of our website instances to validate the installation of the web server and the correctness of the returned response.

But, we couldn’t access the content of those websites publicly from a browser.

Application Load Balancer

For that reason, and also because we want to expose a single endpoint to our end users to access our website servers (and of course balance the load between the instances), we are going to create an Application Load Balancer.

We then have to define a TargetGroup and provide information about its target types (in our case they are going to be EC2 instances), as well as the health checks that it will do to those types.

Finally, we have to associate those two, by adding a listener to the Load Balancer, forwarding all requests to the created Target Group.

All requests made to ALB’s port 80, will be forwarded to the Target Group created above

With the addition of the ALB, we can access the content of our application by hitting the DNS name assigned to the ALB, and the load would be balanced between the 2 servers in a round robin mode (we’ll see how we can do this later on).

Auto Scaling Groups

We want to take things a bit further though, and create an AutoScalingGroup that will be responsible for scaling out/in our application horizontally, based on the criteria that we’ll define.

To do that, we first need to create a definition for the LaunchConfiguration that the ASG will be using to create the instances.

We’ll then write the code for the AutoScalingGroup itself and have it use the above LaunchConfiguration.

As a next step, we define a Scaling Policy for our ASG, which will describe the criteria based on which we want our EC2 instances to scale in or out.

In the specific example, we’ll be using a TargetTrackingScaling policy, checking the number of requests our ALB will be receiving (per target), and scale our target group based on that metric.

Note here, that since from this point onwards the ASG will be responsible for managing the number of website EC2 instances, we no longer need the WebsiteA & WebsiteB resources defined a bit earlier, and you can either comment them out or completely delete them.

If you don’t, by the time that the template finishes running, you will see both them plus the 2 new instances that the ASG will create (it does not count already created instances when checking the DesiredSize attribute).

Metadata

A last small piece that we will add to our template file, is the Metadata section. In this part we’ll just group the Parameters we’ve defined in the beggining of our script, so that they are displayed in a specific way on the user interface.

That’s it, we’re done!

It is now time to run the above file, monitor the sequence of events as resources are created, and test that everything will be working as expected.

Creating the infrastructure

Before we run the file we’ve created, we need to decide on the region that we want to create our infrastructure. In this example we’ll use N. Virginia (us-east-1).

Create a KeyPair

Our first step is to create key-pair file that we will use, in order to ssh into our EC2 instances. This needs to be done before we start running the file, as its value should be passed in the Parameters section.

To do that, we need to:

1. Go to the EC2 service page
2. Click on Key Pairs and then on Create key pair
3. Enter an easy to remember key pair name and click Create

You need to remember the folder that this key pair file was saved locally, and change the permissions of the file depending on whether you are using Linux, Mac OS or Windows.

Run the template file

After we finish with the creation of the key pair file, we need to follow the steps described below:

1. Go to the CloudFormation service page
2. Click on Create Stack
3. Under “Specify Template”, select “Upload a template file”, choose the file we’ve created and click Next
4. Enter a name for the stack to be created
5. In the Parameters section, select the key-pair file previously created and give a value to the SSHLocation field (we can leave the rest with their default values, or change them if we want) and click Next.

6. We leave “Configure Stack Options” and “Advanced Options” as is and we click Next

7. Finally, we review all information regarding the stack, and click Create Stack

We will be redirected to the Stacks page, from which we can monitor the resources being created, the events that trigger their creation, etc.

We can see the order of events that create our infrastructure’s resources

After the completion of our stack, we can navigate to different screens and check the resources created (EC2, VPC, Security Groups, Route Tables, etc).

One thing we want to do, is to go to the Load Balancers section (of the EC2 page), and copy the DNS name of the created LB.

We can then open a web browser and paste that DNS name and we’ll be able to see the response from one of our web servers!

If we keep refreshing the page, we’re able to see that each time we are getting a response from a different web server that is located behind our load balancer.

Cleaning up

After we examine and we play around with the created infrastructure, we need to always remember to DELETE our stack, in order to destroy all created resources! This way we avoid getting charged for resources that we’ll no longer be using.

Hope the above example is helpful, thank you reading!