Simple IaC deployments with dockerized Terraform + IAMLive + Cost Estimation

Have you ever had to rerun your Terraform numerous times until you find out all the permissions it needs to deploy your infrastructure? Had problems setting up IAMs to comply with the principle of least privilege? This article provides an easy to use solution to help you with theses tasks and more!

A Docker image was designed in order to provide an simple environment to work with Terraform and AWS tools (integration with tools from other Cloud Provider are on development!). The image is mainly focused on helping developers with the following challenges:

• Working with multiple projects that use different Terraform versions;
• Making sure all members of a team are working with the exact same Terraform version and environment;
• Knowing exactly what permissions you need to deploy your infrastructure;
• Getting a rough estimate of the costs your infrastructure will result on;

To overcame these challenges, the provided solution:

• Dockerizes Terraform;
• Uses a Makefile, to make it easy to run the dockerized version of Terraform;
• Uses IAMLive (an awesome project by Ian Mckay) to track all AWS permissions used by any command run inside the container (including any Terraform calls);
• Uses terraform-cost-estimation to estimate the cost of a terraform plan;

Instructions on how to use it and how it works are presented on the next sections.

How to use it?

Create docker-compose.yml and Makefile

Using the Rony CI docker image is pretty straight forward. The way we recommend users to do it is by including a docker-compose.ymland a Makefileto the project.

The docker-compose file would look like this:

And the Makefile would look like this:

Setting up your project directory

When using the docker-compose and Makefile provided on the last section, make sure they are in the root folder of your project, and that your terraform files are in the ./infrastructurefolder.

Add your AWS credentials to environment

The docker-compose file pulls AWS credentials from the environment, so you need to set the following environment variables:

• AWS_ACCESS_KEY_ID
• AWS_SECRET_ACCESS_KEY
• AWS_SESSION_TOKEN (only needed when using tools like AWS-vault with MFA)

We recommend doing it by using tools like AWS-vault to securely store the credentials, preferably with multi factor authentication enabled. You could also set the variables manually ( export AWS_ACCESS_KEY_ID=XXX) for example, or create a .env file to be read by docker-compose.

Running Terraform

With your projects directory and the environmental variables set, you can run the dockerized Terraform using the following commands:

• make tf-init: equivalent to terraform init
• make tf-fmt: equivalent to terraform fmt
• make tf-validate: equivalent to terraform validate
• make tf-plan: equivalent to terraform plan
• make tf-apply: equivalent to terraform apply
• make tf-destroy: equivalent to terraform destroy
• make tf-workspace-dev: equivalent to terraform workspace select dev
• make tf-workspace-staging: equivalent to terraform workspace select staging
• make tf-workspace-prod: equivalent to terraform workspace select prod
• make tf-workspace-dev: equivalent to terraform workspace select dev

When running this commands in Linux distributions, you will probably need to run them with administrator privileges. In that case, make sure to run sudo with the -E flag, so it runs with your current environment variables ( sudo -E make tf-init , for example)

After every command you run on the container, your terminal will display the AWS permissions used during the run:

The generated policy, containing the permissions used by Terraform, is also saved to ./terraform/policy.json.

Getting Cost Estimation

Similarly to running terraform, the cost estimation can be obtained by running the command: make tf-plan-cost.

As information is sent to a third party website https://terraform-cost-estimation.com/, and we do not have control on how this information is used, a script is run to remove any sensible information, keeping only the information necessary to estimate costs. All the information to be sent is also displayed to the users. Prior to sending it, the user is required to approve it.

After the command is run, an estimation about the cost of the new infrastructure to be deployed (the resources to be deployed next time terraform apply is run). Keep in mind that this is just an estimation for the cost of resources created by terraform, this resources might create or use, themselves, new resources that would imply in additional costs.

I want to run a different Terraform version

The image is available with different tags for different versions. Have a look on the available ones. Make sure to get the latest version of Rony-CI for the terraform version you want. If you need a version that is not available, let us know through an github issue and we will make it available as soon as possible!

How was it developed?

This project was developed to be part of Rony, an amazing Data Engineering Framework developed by A3Data. It contains a simple Dockerfile, that installs terraform, IAMLive and a tool to make call to https://terraform-cost-estimation.com/.

The entrypoint of the container is set to a script that starts iamlive prior to executing user commands, kills it after it is executed and prints the generated policy to the terminal.

I want to contribute to this awesome project!

If you want to contribute to Rony framework or any of its derived projects, create an issue on the github repository and let us know what you are doing, or any suggestions you might have. We will be glad to help you developing it!

TODO: We also have a Discord channel where we discuss our next steps