Migration of Terraform state between various backends.

Published in

Dr.Max Product Blog

7 min readJun 17, 2022

Sometimes the need to move existing Terraform state from your local machine to remote backend or between remote backends should popup. So how to deal with it easily without affecting existing resources? I'll try to show you how to do it from your command prompt easily.

Imagine the situation when you are developing locally and everything what you've already deployed needs to be moved to the remote state or you would like to switch between two types of remote backends. Sounds familiar? Yes? So in this article I'll guide you how to do it without deleting the resources and spinning them up again with different state configuration.

The example scenario

For this article let's assume simple scenario that we have to create Resource Group and Virtual Network in Azure via AzureRM provider. To achieve stated requirements we have to create following Terraform declaration and as first we will configure Terraform backend to work with the local state file.

Local state

Once we execute terraform init command we can see the following output.

Once the Terraform is initialized you should see the confirmation message. The process also created .terraform folder inside your project. Once we will execute the apply terraform.state file with the actual deployment state will be created in root of the working directory.

TIP: If you need to work from different directory use -chdir flag as is documented here.

Let's execute terraform plan command and verify if plan is contains resources we would like to create. As confirmed by the screenshot it will be exactly as we wanted.

Now we are ready to deploy resources by using terraform apply command. This execution will create the resources via Azure Resource Manager provider and store its state inside local state file on your machine. The output should look similar to this:

We can simply check via Azure portal that resources were created. And terraform.state file with actual deployment was stored on your local file system.

Transferring existing local state to remote backend provided by Azure Storage Account

So now once you are happy with your local deployment and you overcame all the bad ideas — how to manage state locally or via version control systems — It's time to choose one of the possibilities and share remote state properly with your teammates.

Let's extend our example declarations so that we can use remote backend and pick Azure Storage Account as the first option. For that purpose we have to extend our template in following way.

TIP: Each backend configuration key can be passed via CLI parameters as is documented here.

As you can see not much has changed

To make it work we also have to create Storage account and state container inside the Azure. So let's do it via following az cli commands.

Once we have it, we can execute re-initialization of the backend viaterraform init command.

Oops, we forgot to add the -migrate-state flag

This will tell the Terraform that our backend has be changed from local to remote and needs to be migrated first and we can modify our existing resources with state stored on the remote backend.
Let's add the flag and see if it works:

Terraform has asked if we want to copy existing state — to which we respond

yes— to confirm state migration or
no — staring with with an empty state.

TIP: If you choose the no option Terraform will create empty state and all existing, ie. already deployed resources need to be imported to the state via import procedure as is described here.

If you choose yes state will be copied into the remote location and can be used by your teammates as the source of truth. For verification we can check if the state file is visible trough the Azure portal inside defined container:

We can see the state was properly migrated into the Azure Blob Storage Container under defined Blob name and can be used without the need to redeploy or import the resources. Let's test it by changing CIDR of the virtual network to 10.100.0.0/16.

Once we have modified our declaration we can execute terraform plan and terraform apply with following result.

Terraform will do exactly what was planned and after the apply we can once again verify virtual network was modified and state file was updated.

Pretty simple, huh? But what about the other use case — movement between two remote states?

Movement between Azure Storage Account and Terraform Cloud remote states

So let's pretend that we are not satisfied with the Azure Storage Account remote backend features and we'd like to use additional Terraform Cloud features for remote state management.

TIP: If you are not familiar with remote state management and its feautures have a look here.

So let's start with the fresh local directory where we have only the Terraform declaration template and start with the whole process from the scratch. So let's define following structure and main.tf file content.

Now you have to execute the terraform init command to get all the references to your working directory.

Now we’ll do the slight modification to use Terraform Cloud as the remote backend in main.tf file.

To make it work and be able to migrate state we have to create the Terraform Cloud Workspace inside selected Organization (in my case it will be my private one marley-ma). So let's do it.

TIP: For the authentication against the Terraform cloud several options could be used. I'm using the .terraformrc file inside my project folder with user token as is documented here but you can look for the option fitting you needs in the documentation.

Now we are prepared to execute terraform init -migrate-state state to start the migration. Once again we’ll choose the yes option, state will be migrated

and we can see it's the same flow as in the previous case but in different backend.

So as the last bit let's change the template one last time and rename virtual network to hope-you-like-it string. To achieve it we have to modify main.tf file in following way.

Now we can run terraform plan and terraform apply and verify if it works against remote state in the Terraform Cloud Workspace where we have migrated our state.

Conclusion

So as you have seen it was pretty simple. There are just few rules to follow when transgerring your existing state between local or remote state backends.

With this approach you can avoid affecting the existing resources by the movement itself and easily continue with the different backend provider on stuff which was already build with different state management.

There are few topics which I didn't touch in this article — such as Terraform workspaces, remote state prefixes, authentication and so on. But we will cover these separately in one of my future articles.