Part 8— HumanGov Application — Terraform-6: Terraform Remote State

7 min readOct 16, 2023

HumanGov is a software-as-a-service (SaaS) cloud company that will create a Human Resources Management SaaS application for the Department of Education across all 50 states in the US and host the application files and databases in the cloud.
In this following project series, we are going to transition the architecture from a traditional virtual machine architecture to a modern container-based architecture using Docker containers and Kubernetes running on AWS. In addition, we will also be responsible for automating the complete software delivery process using Pipelines CI/CD using AWS services such as AWS CodeCommit, AWS CodePipeline, AWS CodeBuild, and AWS CodeDeploy. Finally, we will learn how to monitor and observe the cloud environment in real-time using tools such as Prometheus, Grafana, and automate one-off cloud tasks using Python and the AWS SDK.
In this section, we are going to introduce working with Terraform Remote State and practice Terraform with cloud provider AWS before using it in the implementation of the HumanGov application. This is the 8th part of a project series. Check Part 1, Part 2, Part 3, Part 4, Part 5, Part 6 and Part 7 to follow up.

Terraform Remote State

Terraform state file records the current state of Terraform infrastructure. Terraform uses state files to keep track of the resources and store their current state file locally. Storing Terraform state files locally is often used if the infrastructure is small, or collaboration on the same infrastructure management isn’t a concern. Local state storage also can be faster. On the other hand, it does not allow working collaboratively. If local state files are lost or corrupted, it can be challenging to recover the state of the infrastructure. This is a big problem, especially for projects with big and complex infrastructure. Additionally, local storing state files poses a security risk because they can contain sensitive information, such as access keys or passwords.

We can store Terraform’s state files in a remote location. It is also called backend. This backend could be an S3 bucket, Azure Storage Account, Google Cloud Storage, Alibaba Cloud OSS, Terraform Cloud, and many others. State locking and versioning become important when using remote state storage in Terraform. State locking prevents concurrent modification of the infrastructure ensuring that only one user or process can make changes at a time. GitHub and GitLab are not recommended for remote storage because they do not support state locking for now. Storing state files remotely also allows versioning which is crucial for recovering using the previous version of it if we have any issues in the current state.

Hands-on: Terraform Remote State

Step 1: Create a new directory for the Terraform project and create a main.tf file in that directory

Go to AWS Services and open Cloud9. Create a project directory named terraform-remote-state-example and main.tf file in the project directory.

mkdir terraform-remote-state-example && cd terraform-remote-state-example
touch main.tf

Step 2: Provide AWS provider and EC2 resource in your main.tf file

We will use the main.tf file to configure our resources for this example project. We will provide provider and EC2 instance resources. To find out the AMI for your resource: go to AWS Services -> EC2 -> Instances -> Launch Instance -> Select Linux -> Copy and paste the AMI ID in main.tf file.

provider "aws" {
  region = "us-east-1"
}

resource "aws_instance" "example" {
  ami           = "ami-041feb57c611358bd"
  instance_type = "t2.micro"
}

Step 3: Initialize and apply your Terraform configuration

terraform init

As you can see, it says “Initializing the backend…”. In this case, our backend is local.

Apply configuration to create an EC2 instance.

terraform plan
terraform apply

Our state file is created including the latest configuration information we just did and store locally in the Cloud9 environment.

Step 4: Create a new S3 bucket to store the Terraform state remotely

Once we move the state file, terraform.tfstate, from a local environment to the S3 bucket anyone who has permission can reach the file and make changes to the Terraform configuration.

We are going to create an S3 bucket separately from the Terraform configuration because if we destroy resources or the configuration we will also destroy the state file. Therefore, we will create an S3 bucket using CLI to keep the state file safe in the S3 bucket.

aws s3api create-bucket --bucket state-file-remote-storage-ctx91 --region us-east-1

Check AWS Services -> S3 bucket to see if it is created.

Step 5: Create a DynamoDB table, terraform-lock-table, to store the terraform.tfstate.lock.info file

In Terraform, the terraform-lock-table is a special DynamoDB table used for state locking when using the S3 remote backend to ensure that only one user can make changes to infrastructure at a time.

We create a DynamoDB table with specific attributes recommended by Terraform:

aws dynamodb create-table \
  --table-name terraform-lock-table \
  --attribute-definitions AttributeName=LockID,AttributeType=S \
  --key-schema AttributeName=LockID,KeyType=HASH \
  --provisioned-throughput ReadCapacityUnits=5,WriteCapacityUnits=5 \
  --region us-east-1

Terraform will manage locks in the terraform-lock-tablewhen using the S3 backend with DynamoDB locking. Additionally, make sure the IAM role or user interaction with DynamoDB table has the necessary permissions to read and write to the terraform-lock-table.

Go to AWS Services -> DynamoDB -> Tables to check.

Our DynamoDB table is empty so far because we did not do any configuration so far.

Step 6: Create a backend.tf file in the project directory

A backend.tf file is used to separate the backend configuration from the main Terraform configuration. It specifies whether the Terraform state is stored locally or in a remote backend. In our case, we are going to use an S3 bucket specifying the details of the S3 bucket.

touch backend.tf

terraform {
  backend "s3" {
    bucket         = "state-file-remote-storage-ctx91"
    key            = "terraform.tfstate"
    region         = "us-east-1"
    encrypt        = true
    dynamodb_table = "terraform-lock-table"
  }
}

backend "s3": Specifies remote backend type. We can replace this with "azurerm" for Azure, "gcs" for Google Cloud Storage, or "remote" for Terraform Cloud
bucket: The name of the S3 bucket
key: The name of the state file that we want to store
region: The AWS region in which the S3 bucket is located
encrypt: Specifies whether state file data is encrypted
dynamodb_table: Specifies the name of the DynamoDB table to use for state-locking

Step 7: Initialize Terraform

Once we initialize Terraform it can understand from the backend.tf file that we will store the state file remotely in the S3 bucket.

Run terraform init that starts migration of the state file from the local environment to the remote S3 bucket backend.

During the initialization, Terraform will prompt you to confirm the migration of the local state to the remote state. Type yes and press Enter to confirm.

Step 8: Apply the Terraform configuration

terraform apply

Now, Terraform will store the state remotely in the specified S3 bucket. We will still have the state file locally but it will be empty. You can verify this by checking the contents of the S3 bucket using the AWS CLI or the AWS Management Console.