Terraform — moving resources around… in PROD

Problem

Recently I wrote an article about pros of converting count to for_each: ‘Terraform for_each vs count’ — this is all cool however if you ‘just do it’ it will re-create your resources. In non-productive that might be ok, but in Production / Live environments this can be potentially disastrous.

Solution

If you’d like to convert existing resource from count to for_each or actually move a resource between various states (yes, I did that a few times too!) without actually destroying and re-creating it keep reading!
A fairly simple solution is to alter state via terraform state subset of (apparently advanced) commands.
In short words, what you need to do is simply remove an existing resource from state and re-import it with correct ID — that’s all! Sounds simple? It actually is!

How to actually do it

!DISCLAIMER! This can be potentially dangerous! Always backup you state BEFORE doing this as recovery can be tricky if something went really wrong!

So I will explain, step by step the whole process.
1. Initial setup with count:

variable "my_list" {
  default = ["first", "second"]
}
resource "google_compute_project_metadata_item" "medium_example" {
  count = length(var.my_list)
  key   = var.my_list[count.index]
  value = var.my_list[count.index]
}

Above setup produces following plan:

Terraform will perform the following actions:
# google_compute_project_metadata_item.medium_example[0] will be created
  + resource "google_compute_project_metadata_item" "medium_example" {
      + id      = (known after apply)
      + key     = "first"
      + project = (known after apply)
      + value   = "first"
    }
# google_compute_project_metadata_item.medium_example[1] will be created
  + resource "google_compute_project_metadata_item" "medium_example" {
      + id      = (known after apply)
      + key     = "second"
      + project = (known after apply)
      + value   = "second"
    }
Plan: 2 to add, 0 to change, 0 to destroy.

Apply above plan (create those resources).

2. Change count based resource to for_each based one, state file after conversion will look like this (please refer to for_each vs count article for details on that):

variable "my_list" {
  default = ["first", "second"]
}
resource "google_compute_project_metadata_item" "medium_example" {
  for_each = toset(var.my_list)
  key   = each.key
  value = each.value
}

As a result of this change we will get following plan:

Terraform will perform the following actions:
# google_compute_project_metadata_item.medium_example will be destroyed
  - resource "google_compute_project_metadata_item" "medium_example" {
      - id      = "first" -> null
      - key     = "first" -> null
      - project = "MY_PROJECT" -> null
      - value   = "first" -> null
    }
# google_compute_project_metadata_item.medium_example[1] will be destroyed
  - resource "google_compute_project_metadata_item" "medium_example" {
      - id      = "second" -> null
      - key     = "second" -> null
      - project = "MY_PROJECT" -> null
      - value   = "second" -> null
    }
# google_compute_project_metadata_item.medium_example["first"] will be created
  + resource "google_compute_project_metadata_item" "medium_example" {
      + id      = (known after apply)
      + key     = "first"
      + project = (known after apply)
      + value   = "first"
    }
# google_compute_project_metadata_item.medium_example["second"] will be created
  + resource "google_compute_project_metadata_item" "medium_example" {
      + id      = (known after apply)
      + key     = "second"
      + project = (known after apply)
      + value   = "second"
    }
Plan: 2 to add, 0 to change, 2 to destroy.

Not really cool, Terraform wants destroy and re-create our resources!

3. Remove and re-import resources into state (please note medium breakes lines in example below!)
INFO: terraform state rm <resource> without index will remove ALL sub-elements of an indexed element.
INFO2: When importing key indexed element we need to escape “ characters.

$ terraform state rm google_compute_project_metadata_item.medium_example
Removed google_compute_project_metadata_item.medium_example[0]
Removed google_compute_project_metadata_item.medium_example[1]
Successfully removed 2 resource instance(s).
$ terraform import google_compute_project_metadata_item.medium_example[\"first\"] first
google_compute_project_metadata_item.medium_example["first"]: Importing from ID "first"...
google_compute_project_metadata_item.medium_example["first"]: Import prepared!
  Prepared google_compute_project_metadata_item for import
google_compute_project_metadata_item.medium_example["first"]: Refreshing state... [id=first]
Import successful!
$ terraform import google_compute_project_metadata_item.medium_example[\"second\"] second
google_compute_project_metadata_item.medium_example["second"]: Importing from ID "second"...
google_compute_project_metadata_item.medium_example["second"]: Import prepared!
  Prepared google_compute_project_metadata_item for import
google_compute_project_metadata_item.medium_example["second"]: Refreshing state... [id=second]
Import successful!
$ terraform plan
Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.
google_compute_project_metadata_item.medium_example["first"]: Refreshing state... [id=first]
google_compute_project_metadata_item.medium_example["second"]: Refreshing state... [id=second]
------------------------------------------------------------------------
No changes. Infrastructure is up-to-date.

So, what actually happened? Well, we removed our count based resources and imported the same actual resources as for_each based ones. Difference between two is that one is indexed based on list index and second on map key.

Summary

I hope above example helps to understand the process at least a bit! If you have questions let me know via comments.

Also, have a look at my other Terraform articles on medium!