How to manage Terraform state in Azure Blob Storage

Photo by chris panas on Unsplash

Every time you ran terraform plan or terraform apply, Terraform was able to find the resources it created previously and update them accordingly. But how did Terraform know which resources it was supposed to manage?

Whenever you run terraform apply it creates a file in your working directory called terraform.tfstate. The State is an essential building block of every Terraform project. It will act as a kind of database for the configuration of your terraform project. Terraform uses this local state to create plans and make changes to your infrastructure. Prior to any operation, Terraform does a refresh to update the state with the real infrastructure.

This is how a tfstate file looks like. In this state I have just created a new resource group in Azure.

terraform.tfstate

Terraform workflow

This diagram explains the simple workflow of terraform. The .tfstate file is created after the execution plan is executed to Azure resources. Terraform destroy command will destroy the Terraform-managed infrastructure, that too terraform understands from the .tfstate file.

Why state needs to be managed and locked?

It might be okay if you are running a demo, just trying something out or just getting started with terraform. However, in real world scenario this is not the case. Because your laptop might not be the truth for terraform, If a colleague now ran terraform plan against the same code base from their laptop the output would be most likely incorrect.

Terraform also creates a file lock on the state file when running terraform apply which prevents other terraform executions to take place against this state file. With local state this will not work, potentially resulting in multiple processes executing at the same time.

Introducing Terraform Backend

Terraform Backends determine where state is stored. For example, the local (default) backend stores state in a local JSON file on disk. The Consul backend stores the state within Consul. Both of these backends happen to provide locking: local via system APIs and Consul via locking APIs. There are a number of supporters for backend — s3, artifactory, azurerm, consul, etcd, etcdv3, gcs, http, manta, terraform enterprise etc..

In this article we will be using Azurerm as the backend. It Stores the state as a Blob with the given Key within the Blob Container within the Azure Blob Storage Account. This backend also supports state locking and consistency checking via native capabilities of Azure Blob Storage.

Lets see how can we manage Terraform state using Azure Blob …

Create a Storage Account

Here I am using azure CLI to create azure storage account and container.

Create Azure storage account

Configure State Backend

Now we have an instance of Azure Blob Storage being available somewhere in the cloud; Different authentication mechanisms can be used to connect Azure Storage Container to the terraform backend — Azure CLI or Service Principal, Managed Service Identity, Storage Account Access Key, Storage Account associated SAS Token.

1. Using Service Principal

terraform {   
 backend "azurerm" {     
 storage_account_name  = "tstate09762"     
 container_name        = "tstate"     
 key                   = "terraform.tfstate"   
 } 
}

2. Using Managed Service Identity

terraform {   
 backend "azurerm" {     
 storage_account_name  = "tstate09762"     
 container_name        = "tstate"     
 key                   = "terraform.tfstate"
 use_msi               = true
 subscription_id       = "00000000-0000-0000-0000-000000000000"
 tenant_id             = "00000000-0000-0000-0000-000000000000"   
 } 
}

If the Backend is configured, you can execute terraform apply once again. Terraform will ask if you want to push the existing (local) state to the new backend and overwrite potential existing remote state. After answering the question with yes, you’ll end up having your project migrated to rely on Remote State.

The backends key property specifies the name of the Blob in the Azure Blob Storage Container which is again configurable by the container_name property.

State Locking

State locking is used to control write-operations on the state and to ensure that only one process modifies the state at one point in time. Not all State Backends support state locking. Luckily it’s supported for Azure Blob Storage by using the previously referenced Azure Blob Storage Lease mechanism. State locking is applied automatically by Terraform.

Manage versions with snapshots

Blob storage service has the ability to create snapshots of the blobs that can be used for tracking changes done on a blob over different periods of time. Snapshots provide an automatic and free versioning mechanism. Using snapshots, you can rollback any changes done on a blob to a specific point in time or even to the original blob. Using this feature you can manage the version of your state file.

You can still manually retrieve the state from the remote state using the terraform state pull command. This will load your remote state and output it to stdout. You can choose to save that to a file or perform any other operations.

Useful Links

Terraform state docs, backend docs, backends: azurerm

https://www.slideshare.net/mithunshanbhag/terraform-on-azure-166063069

If you are new to Terraform and IaC you can start with — Getting Started with Terraform and Infrastructure as Code

Follow us on Twitter 🐦 and Facebook 👥 and join our Facebook Group 💬.

To join our community Slack 🗣️ and read our weekly Faun topics 🗞️, click here⬇

If this post was helpful, please click the clap 👏 button below a few times to show your support for the author! ⬇