Further Terraform enhancements

Embracing a few simple changes to improve the simplicity and resilience of our Terraform solution.

Jon Povey
Jon Povey
Mar 12 · 5 min read
Source: Pixabay

Terraform is a powerful tool developed by HashiCorp to declaratively define immutable infrastructure-as-code. In my last post Bringing Terraform under control, I described how my team and I had harnessed the power of Terraform to organise our infrastructure and keep the complexity of our solution to a minimum. Since then, we’ve made a number of changes to further improve upon this.

Simplifying environments

Example of previous environment folder structure

The reason we started with one file per environment and region was to allow us to set specific variables and configure the Terraform backend to store state.

Example of the previous environment and region specific main.tf

In the new design, the death by folders environment structure is gone and each environment is represented by a single JSON file. This means that specific variables can still be set and the entry point is now a single main.tf in the project route which makes things a lot clearer.

Example of the new environment agnostic main.tf in project root
Environment specific variables represented using json

As our infrastructure is the same in each region there is also no need for region specific variables which further reduces duplication. The region was already being passed into the go.ps1 script, to specify the Azure region when running the Terraform commands in the build pipeline, so this parameter can now also be passed into the Terraform plan as a variable to create unique resource names based on our naming conventions.

Example of new environment folder structure

This change was made possible by shifting the responsibility of initialising the backend from the environment main.tf file to the go.ps1 script. The Terraform init command provides functionality to set up the backend using parameters which we extract from the environment JSON.

Executing terraform init using parameters

The Terraform plan is able to accept JSON as a variable file, so that same environment JSON that’s used to extract the backend parameters can be used in the plan. Having a single entry point guarantees the same blueprints are applied to every environment. These changes also allowed us to remove over 250 lines of code and make it easier to maintain, thanks to reduced duplication.

Executing Terraform plan using a variable file

Terraform does provide a concept called workspaces which can be used to control variables for different configurations. I personally feel that workspaces can quickly become quite cluttered, especially if you need to maintain a number of environments. I also don’t think they work so well in automation so we decided to go with the approach above.

Set secrets using Terraform commands

Setting up the provider using PowerShell secret replacement

Now that we have harnessed the ability to run the Terraform commands with variables at runtime, there’s no longer a need to replace and set the keys using PowerShell. The secret is still passed into the go.ps1 script but it is instead passed in as a Terraform variable which makes the script less complex and more secure.

Setting up the provider using variables

Terraform state backup

The state, however, is not backed up, so users are unable to view the history of state, or, retrospectively fix state that has been modified incorrectly (this is unlikely but it’s always good practice to have a disaster-scenario plan). In order to combat this, we now manually backup the state to a separate location before running Terraform apply.

This is simply achieved by using the built-in Azure PowerShell commands to copy the state to an alternate storage account. And, as we already have the credentials set in the variable files, it makes it easy to authenticate.

Hopefully this article provided you with some ideas to help simplify your Terraform solution — however big or small the change is.

Feel free to drop me a message or give me some claps so more people can see this post.

The ASOS Tech Blog

A collective effort from ASOS's Tech Team, driven and directed by our writers. Learn about our engineering, our culture, and anything else that's on our mind.

Jon Povey

Written by

Jon Povey

Software Engineer @ASOS

The ASOS Tech Blog

A collective effort from ASOS's Tech Team, driven and directed by our writers. Learn about our engineering, our culture, and anything else that's on our mind.

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade