How to launch the Autoscaling group with remote state management using Amazon-S3. Terraform Basics part:2
In the previous blog post How to Launch EC2 instance using Terraform. Terraform Basics part:1, we have discussed about all Terraform basics including the installation of Terraform, Managing the state file locally and we have also launched one EC2 instance.
In this post, we will be discussing how to manage state file with a remote backend and we will also be launching a cluster of web servers with the help of an auto-scaling group.
What is Terraform state?
In the previous post, we have discussed about the state file and how the Terraform will use that state file to track the provisioned infrastructure. Last time when we ran the commands terraform apply and terraform planif you have noticed how the terraform was able to recall the resources it created. Now, what we will do is we will store the state file in remote backend such as Amazon s3. So that our terraform will fetch the state information from the remote backend, which helps more than one developer to work on the same Terraform configuration files.
To do so create one S3 bucket and DynamoDB table. You may ask a question that we are creating an s3 bucket to store the state management file but why are we creating a DynamoDB table. What is the use of it?
Before I answer this question I want to talk about a situation where a group of developers will be working on the same Terraform configuration files. What if more than one developer is working on the same resource. This problem will lead to resource conflicts and the provisioned infrastructure may get disturbed. Terraform has come up with a solution for this problem called State-Lock. To run terraform apply, Terraform will automatically acquire a lock; if someone else is already running the command terraform apply, they will already have the lock, and you will have to wait. You can run apply with the -lock-timeout=<TIME> parameter to tell Terraform to wait up to TIME for a lock to be released (e.g., -lock-timeout=10m will wait for 10 minutes). The Amazon S3 supports locking via DynamoDB. DynamoDB supports very important features such as conditional writes and strongly-consistent reads that are required for a distributed lock system.
Follow the below file structure in order to create S3 and DynamoDB table:
<Root directory>
├── main.tf
├── outputs.tf
└── variables.tfYou can find the required code to create S3 and DynamoDB here.
Open the file main.tf and write the following code to add provider aws :
provider "aws" {
region = "us-east-2
}In order to create an S3 bucket write the following code in the file main.tf :
resource "aws_s3_bucket" "terraform_state" { #bucket name should be globally unique bucket = "terraform-up-and-running-state-4567" # Enable versioning so we can see the full revision history of our # state files
versioning {
enabled = true
}
# Enable server-side encryption by default server_side_encryption_configuration {
rule {
apply_server_side_encryption_by_default = {sse_algorithm = "AES256"
}}
}
}
“aws_s3_bucket” is the resource name, terraform_state is the identifier we are using for this resource. This identifier is used to call the s3 bucket resource in Terraform configuration files.bucket is where we will give the s3 bucket resource a name that can be used in the AWS Console. Make a note that the bucket name should be globally unique. versioning is a feature provided by s3. For example, If you are working on a Terraform configuration file with the state storing in s3. If you make any changes to the provisioned infrastructure then the state file in the s3 will be replaced with a new state file but the old state file will be stored in the version history. So that if you want to roll back to the oldest state file then you can collect it from the version history. Here we are enabling the version by assigning a boolean value true .
server_side_encryption_configuration will handle the encryption part of the state file. Terraform will not come with a built-in feature to encrypt the state file. So s3 supports server_side_encryption . However, Terraform uses the protocolHTTPS to send the state file to s3 . One need not worry about the encryption_at_transit . Checkout the Terraform documentation on S3.
So far we have written code for s3. Now we will write code to create a DynamoDB table. Create a DynamoDB table with LOCKID a primary key. aws_dynamo_table resource can be used to create a DynamoDB table:
resource "aws_dynamodb_table" "terraform_locks" { name = "terraform-up-and-running-locks" billing_mode = "PAY_PER_REQUEST"
hash_key = "LockID"
attribute {
name = "LockID"
type = "S"
}
}Save the code and run the commandterraform init to download the provider code. After the provider code is downloaded, before you run the command terraform apply . I want you to open the outputs.tf file and write the following in it.
output "s3_bucket_arn" {
value = aws_s3_bucket.terraform_state.arn
description = "The ARN of the S3 bucket"
}output "dynamodb_table_name" {
value = aws_dynamodb_table.terraform_locks.name
description = "The name of the DynamoDB table"
}
As you can see the output variable s3_bucket_arn is used to find out the arn of created s3 bucket.
Output variable dynamodb_table_name gives you the name of the DynamoDB as table created.
Now run the command terraform apply to launch the resources. You may see output as follows:
An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
+ createTerraform will perform the following actions:
..................Plan: 2 to add, 0 to change, 0 to destroy.Do you want to perform these actions?
Terraform will perform the actions described above.
Only 'yes' will be accepted to approve.Enter a value: yesaws_dynamodb_table.terraform_locks: Creating...
aws_s3_bucket.terraform_state: Creating...
aws_dynamodb_table.terraform_locks: Still creating... [10s elapsed]
aws_s3_bucket.terraform_state: Still creating... [10s elapsed]
aws_dynamodb_table.terraform_locks: Creation complete after 13s [id=terraform-up-and-running-locks]
aws_s3_bucket.terraform_state: Still creating... [20s elapsed]
aws_s3_bucket.terraform_state: Still creating... [30s elapsed]
aws_s3_bucket.terraform_state: Creation complete after 33s [id=terraform-up-and-running-state-4567]Apply complete! Resources: 2 added, 0 changed, 0 destroyed.Outputs:dynamodb_table_name = terraform-up-and-running-locks
s3_bucket_arn = arn:aws:s3:::terraform-up-and-running-state-4567
As you can see dynamodb_table_name = terraform-up-and-running-locks and s3_bucket_arn = arn:aws:s3:::terraform-up-and-running-state-4567
Open the console and check whether the resources are created or not.
As you can see both resources are up and running.
You have successfully created an s3 bucket and a DynamoDB table required for remote-backend state.
Now let us start with the Terraform configuration code to create a cluster of web-servers using auto-scaling-group.
Come out from the root directory, and create one more directory with the name terraform_autoscaling_group
Follow the below file structure:
terraform_autoscaling_group
├── main.tf
├── outputs.tf
└── variables.tfRequirements:
- Create the resource
aws_launch_configuaration - Create the resource
aws_autoscaling_groupand attachaws_launch_configuarationto it. - Create the resource
aws_elband attach to theaws_autoscaling_group
You can get the code reference here.
Create AWS launch configuration:
The first thing you have to do is add provider in main.tf and declare a variable in varibale.tf
To make the infrastructure code re-usable we must use variables. Write the following code in the file variables.tf :
variable "server_port" {
description = "The port the server will use for HTTP requests"
type = number
default = 8080
}Note: Remember to replace the value 8080 with ${var.server_port}
To create an AutoScaling we need a resource called aws_autoscaling_group. To create an auto-scaling group you must specify a launch_configuration. To do so write the following code in the same file to create the resource aws_launch_configuration .
resource "aws_launch_configuration" "example"{
image_id = "ami-0c55b159cbfafe1f0"
instance_type = "t2.micro"
user_data = <<-EOF
#!/bin/bash
echo "Hello, World" > index.html
nohup busybox httpd -f -p "${var.server_port}" &
EOF
lifecycle {
create_before_destroy = true
}
}In the previous post, we have created one EC2-Instance runs a web server on a port 8080 that displays the message Hello world. Now our goal is to create a group of web servers with the same functionality. To do so we register aws_launch_configuration resource to aws_autoscaling_group resource. So that the aws_autoscaling_group will launch Ec2 instances with the launch_configuration given by aws_launch_configuration resource.
The attribute lifecycle is nothing but a deployment strategy that autoscaling_group will create a new instance before deleting the old instances.
Our instances need security_group to control the incoming and outgoing traffic of instances. The resource type aws_security_group provides a security group resource. To do so add the following code to the file main.tf.
resource "aws_security_group" "instance" {
name = "terraform-example-instance"
ingress {
from_port = var.server_port
to_port = var.server_port
protocol = "tcp"
cidr_blocks = ["0.0.0.0/0"]
}
}To create a security group from_port , protocol and to_port are mandatory arguments in ingress.
Now to attach this security group to launch configuration resource. Add this code in aws_launch_configuration resource with the help of attribute referencing.
security_groups = [aws_security_group.instance.id]Create AWS Autoscaling Group(ASG):
Now, define the resource aws_autoscaling_group by adding the following code to the file main.tf :
resource "aws_autoscaling_group" "example" {
launch_configuration = aws_launch_configuration.example.id
min_size = 2
max_size = 10
tag {
key = "Name"
value = "terraform-asg-example"
propagate_at_launch = true
}}
launch_configuration = aws_launch_configuration.example.id this line registers the created resourceaws_launch_configuration with the resourceexample aws_autoscaling_group.
You may have a doubt that we have not defined in which availability_zones the instances will be launched. In a region, there will be multiple availability zones ranging from 10 to 20. It is not recommended to hard-code the availability zones. Solving this problem Terraform has come up with a feature called Data Sources.
Data sources are used to fetch data and to use in Terraform configuration.
The data block creates a data instance of the given TYPE (first parameter) and NAME (second parameter).
Generally, we use data sources to fetch a list of AMI’s, availability zones for a specified region in AWS.
data "aws_avalability_zones" "all" {} add this line to the filemain.tf .
The syntax of data source is:
data <"resource_name"> "identifier"{
[config...]
}To use this data source in aws_autoscaling_group resource add the following line to aws_autoscaling_group :
resource "aws_autoscaling_group" "example" {
launch_configuration = aws_launch_configuration.example.id
availability_zones = [data.aws_availability_zones.all.id]
min_size = 2
max_size = 10
tag {
key = "Name"
value = "terraform-asg-example"
propagate_at_launch = true
}}
The line availability_zones = [data.aws_availability_zones.all.id] represents a list of availability zones are assigned to availability_zones attribute.
Create Elastic Load Balancer(ELB):
Elastic load balancer will receive the traffic from the internet and it will share the traffic to multiple EC2-instances across availability zones.
The Load Balancers are of three types:
- Application Load Balancer — is used for HTTP and HTTPS requests.
- Network Load Balancer — is used for TCP and UDP requests.
- Classic Load Balancer- is the old one and less efficient than Application and Network Load Balancers.
As of now, we will go with Classic Load Balancer. The resource type aws_elb creates Elastic Load Balancer resource, also known as Classic Load Balancer.
To create Classic Load Balancer write the following code in the file main.tf
resource "aws_security_group" "elb_sg" {
name = "terraform-example-elb-sg"
#Allow all outbound
egress {
from_port = 0
to_port = 0
protocol = "-1"
cidr_blocks = ["0.0.0.0/0"]
}
# Inbound HTTP from anywhere
ingress {
from_port = 80
to_port = 80
protocol = "tcp"
cidr_blocks = ["0.0.0.0/0"]
}}
resource "aws_elb" "example" {
name = "terraform-asg-elb-example"
availability_zones = data.aws_availability_zones.all.names
security_groups = [aws_security_group.elb_sg.id]
# This adds a listener for incoming HTTP requests.
health_check {
target = "HTTP:${var.server_port}/"
interval = 30
timeout = 3
healthy_threshold = 2
unhealthy_threshold = 2
}
listener {
lb_port = 80
lb_protocol = "http"
instance_port = var.server_port
instance_protocol = "http"
}}
example is the identifier for the resource aws_elb . The attributehealth_check is that we are telling the Load Balancer to conduct health checks on targeted instances at interval of 30 seconds. listener says that the load balancer should listen to only HTTP requests at the port 80.
The load balancer also needs a security group. In the above code, we have also created one security group with the name elb_sg . With the help of this line in the load balancer, security_groups = [aws_security_group.elb_sg.id] we have attached the security group to the load balancer.
To create a security group from_port , protocol and to_port are mandatory arguments in ingress and egress .
The attribute egress controls the outbound traffic and ingress controls the inbound traffic. protocol="-1" means we are allowing all kinds of traffic to go out from the security group.
You need to tell aws_elb that you have to send traffic to an autoscale_group . To do so we have to register this aws_elb to aws_autoscaling_group .
To do so, add this line load_balancers = [aws_elb.example.name] to aws_autoscaling_group resource.
resource "aws_autoscaling_group" "example" {
launch_configuration = aws_launch_configuration.example.id
load_balancers = [aws_elb.example.name]
availability_zones = [data.aws_availability_zones.all.id]
min_size = 2
max_size = 10
tag {
key = "Name"
value = "terraform-asg-example"
propagate_at_launch = true
}}
So far we have completed:
- Creating a
aws_launch_configuration - Creating an
aws_autoscaling_groupand attached theaws_launch_configurationto it. - Creating a
aws_elband registered withaws_autoscaling_group
Save the code and run the command terraform init . This command will download all the providers. Then run the command terraform apply .
Your output may look like this:
terraform plan
Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.data.aws_availability_zones.all: Refreshing state...------------------------------------------------------------------------An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
+ createTerraform will perform the following actions:
..................................
........................Apply complete! Resources: 5 added, 0 changed, 0 destroyed.Outputs:clb_dns_name = terraform-asg-elb-example-1636328898.us-east-2.elb.amazonaws.com
As you can see the resources have been successfully created and we got the output value of dns_name for elb:
clb_dns_name = terraform-asg-elb-example-1636328898.us-east-2.elb.amazonaws.comcheck with the AWS Console for the resources.
As you can see there are two instances up and running successfully.
Try to curl the elb dns:
curl http://terraform-asg-elb-example-1636328898.us-east-2.elb.amazonaws.com
Hello, WorldTry to terminate one of the instances and ping the server you will get the same result.
So far, you have successfully created a cluster of web servers using the autoscale group and load balancer. But our main goal is to put of state management file in remote backend s3 bucket.
To configure Terraform to store the state in your S3 bucket (with encryption and locking), you need to add a backend configuration to your Terraform code. This is the configuration for Terraform itself, so it lives within a terraform block, and has the following syntax:
terraform {
backend "<Backend_name>" {[config...]
}
}
Where Backend_name is the name of the backend you want to use (e.g., "s3") and config consists of one or more arguments that are specific to that backend (e.g., the name of the S3 bucket to use). Here’s what the backend configuration looks like for an S3 backend:
terraform {
backend "s3" {
# Replace this with your bucket name!
bucket = "terraform-up-and-running-state-4567"
key = "global/s3/terraform.tfstate"
region = "us-east-2" # Replace this with your DynamoDB table name!
dynamodb_table = "terraform-up-and-running-locks"
encrypt = true
}
}We will cover the code step by step:
bucket is the s3 bucket name we have created in the starting of the post. key is nothing but the path where we want to store our state management file. region says in which region the bucket is created. dynamodb_table here we give the dynamo DB table name for the lock.
Run the command terraform init . To load the state management file into s3 bucket and bring the latest state management file from s3.
Your output may look like this:
Initializing the backend...
Acquiring state lock. This may take a few moments...
Do you want to copy existing state to the new backend?
Pre-existing state was found while migrating the previous "local" backend to the
newly configured "s3" backend. No existing state was found in the newly
configured "s3" backend. Do you want to copy this state to the new "s3"
backend? Enter "yes" to copy and "no" to start with an empty state.Enter a value: yesReleasing state lock. This may take a few moments...Successfully configured the backend "s3"! Terraform will automatically
use this backend unless the backend configuration changes.Initializing provider plugins...The following providers do not have any version constraints in configuration,
so the latest version was installed.To prevent automatic upgrades to new major versions that may contain breaking
changes, it is recommended to add version = "..." constraints to the
corresponding provider blocks in configuration, with the constraint strings
suggested below.* provider.aws: version = "~> 2.54"Terraform has been successfully initialized!You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.
If you notice Acquiring state lock. This may take a few moments… and Releasing state lock. This may take a few moments… . This is exactly what we talked about DynamoDB table for lock management.
As you can see these statements:
Pre-existing state was found while migrating the previous "local" backend to the
newly configured "s3" backend. No existing state was found in the newly
configured "s3" backend. Do you want to copy this state to the new "s3"
backend? Enter "yes" to copy and "no" to start with an empty state.It is asking us for permission to copy the existing state management file to remote backend “s3”.
Check with the AWS Console for S3 status:
As you can see our terraform.state file is created in s3 bucket at the path global/s3/terraform.tfstate .
Cleanup:
Before concluding this blog post I want you to clean up the configured infrastructure by running the command terraform destroy to avoid billing.
It is always recommended to create s3 and DynamoDB in a separate root folder. So that the same backend can be used for multiple projects with different paths setting. And One more important thing is if you want to destroy the s3 and DynamoDB Infrastructure. Only DynamoDB will be deleted. The S3 will throw some error that the version files are still there in the bucket. This is a common-bug in Terraform. Terraform may come up with a solution for this in future versions.
Conclusion
So far in this blog post, we have discussed how to create a cluster of web-servers using load balancer and autoscaling group. And remote backend state management.
In the next blog post, we will discuss how to maintain different workspaces and environments(stage, production and management)in Terraform configuration files.
If you need help with Terraform, DevOps practices, or AWS at your company, feel free to reach out to us at Vitwit.
