Terraform Using AWS S3 Remote Backend

Zeeshan Baig
May 22, 2018 · 4 min read

Overview

These days Terraform is the industry’s go-to tool for Infrastructure automation. Terraform allows you to write infrastructure as a code, which you can manage via source control and one of many benefits is that you can keep track of the changes of your infrastructure (which is a nightmare for any organization).

How Terraform keep track of the changes in your environment? it creates a terraform.tfstate file on a local filesystem. TF state file is simply a small database of the state of your environment. Whenever you run terraform plan, apply or destroy commands it reads the current state from terraform.tfstate file and applies changes to it.

Problem

The problem arrives when you are working in a team. Since terraform.tfstate file is created on your local file system the other developer does not have visibility to it. When any other developer executes the same scripts terraform will create a new terraform.tfstate file which would be different from the current state.

Common solutions to this issue could be to store terraform.tfstate in a source control, that might work in a small team where one person is working at a time or where you have the option of having a different account for each developer. One issue with that is also the .tfstate file could have some sensitive information (such as RDS passwords) that you don’t want to upload to source control systems like GitHub.

Another solution is to use Terraform enterprise solution which comes with all bells and whistles.

In this post, I will show you how you can solve this problem using Remote backends, how can you setup Terraform to use S3 buckets to keep the state of your environment.

Remote Backends

There are many types of remote backends you can use with Terraform but in this post, we will cover the popular solution of using S3 buckets.

Following are some benefits of using remote backends

I hope that gives you enough info on remote backends, let’s dive into the solution.

Pre-requisites

Note: Best practice is to assign policies to a group in my case it is called Developers

Solution

Note: Download example Terraform code from our GitHub repo

The solution to this problem is quite simple you need to add the following code in your .tf file

terraform 
{
backend "s3"
{
bucket="datanextapps"
key="terraform/dev/terraform_dev.tfstate"
region="us-east-1"
}
}

bucket — the name of the S3 bucket in your account

key — the name of .tfstate file in my example I am keeping the file under terraform/dev/ folders inside my datanextapps bucket

region — region of your s3 bucket

To see the running example download this code

Setup AWS access keys and secret keys using aws configure command, optionally replace the values in the code

Execute the following commands from the folder where your main terraform file exists

$ terraform init 
$ terraform plan
$ terraform apply
$ terraform show

If all goes well you will see terraform.tfstate file in your S3 bucket as shown in the slide

Hope you find this post useful.

About DataNext

DataNext Solutions is US based system integrator, specialized in Cloud, Big Data, DevOps technologies. As a registered AWS partner, our services comprise of any Cloud Migration, Cost optimization, Integration, Security and Managed Services. Click here and Book Free assessment call with our experts today or visit our website www.datanextsolutions.com for more info.


Originally published at datanextsolutions.com on May 22, 2018.

Zeeshan Baig

Written by

Cloud, Big Data, DevOps Expert, Musician & CEO of DataNext Solutions, helping people every day with the latest tech. Connect @LinkedIn http://bit.ly/zb-linkedin