Infrastructure as Code — Lists vs Maps and Preserving Order

In Terraform, lists and maps are two different data structures that are commonly used to organize and manage resources.

A typical list variable looks like this:

variable “mylist” {
 description = “An example list variable”
 type = list(string)
 default = [“element1”, “element2”, “element3”]
}

In the above example, a variable called mylist has three values. You can iterate over this variable using aforloop (YES, you can use for loop on the list as well!) and the index of the list element will become the key (in the key-value pair) when a resource will be created. An index of the list always starts with 0 . In other words, the first element in the list is at the zeroth position.

Example of iterating over a list variable:

resource “null_resource” “this” {
 for_each = var.mylist
 
 triggers = {
   key = each.key
   value = each.value
 }
}

In the above code block, the value of each.key for the first iteration will be "0" and value of each.value for the first iteration will be element1 . The value of each.key for the second iteration will be "1" and value of each.value for the second iteration will be element2 . And so on and so forth.

Although, iterating over lists and using the index of the list as a key preserves the order but one problem with this method is that whenever you need to update the list, Terraform will try to recreate all the existing elements of the list as well, even though you add a new element at the end of the list. This is a common problem when the index of the list is used as a key. Let’s park the problem here and I’ll address it at the end of the post.

The second option for organizing data structures is using maps. Below is a basic variable of type map:

variable “mymap” {
 description = “An example map variable”
 type = map(string)
 default = {
 key1 = “value1”
 key2 = “value2”
 key3 = “value3”
 }
}

Now when you iterate over the above map variable, key1 will be the value for the key of the first element and value1 will be the value of the first element.

Example of iterating over a map variable:

resource “null_resource” “this” {
 for_each = var.mymap
 triggers = {
   key = each.key
   value = each.value
 }
}

It looks like this method solves both of the problems, i.e.

1. Terraform will not try to recreate existing elements of the map if it’s updated.
2. Order will be preserved with key1 will be the first resource, key2 the second and key3 the third resource.

Now if you create another map variable that looks like this:

variable “newmap” {
 description = “An example map variable”
 type = map(string)
 default = {
 dev = “value1”
 test = “value2”
 prod = “value3”
 }
}

Do you think Terraform will still preserve the order? The answer is no. Because Terraform does the sorting internally and the first element will be dev , the second element will be prod , and the third element will be test . Now, this is something I don’t want in certain scenarios. Let’s take a look at the solution to this and the problem we parked earlier.

Let’s create a list variable that looks like as shown below:

variable “environments” {
 description = “List of environments”
 type = list(map(string))
 default = [ 
    {
      id = 1
      name = "dev" 
    },
    {
      id = 2
      name = "test" 
    },
    {
      id = 3
      name = "prod" 
    }
  ]
}

I have created a list of maps. Each map further contains multiple string elements. The trick here is that I have provided a custom id parameter that I will use to generate the custom key for Terraform resources instead of using the index of the list elements.

Here is what the code will look like:

resource “null_resource” “this” {
 for_each = { for k, v in var.environments: format("%04d", v.id) => v }
 triggers = {
   key = each.key
   value = each.value.name
 }
}

Let’s break down what’s going on in the above code block:

1. A custom key will be generated using v.id
2. The format() function produces a string and adds the left padding (using %04d ) to the v.id value. The %04d switch generates values like 0001 , 0002 , etc.
3. Why is format() needed? It’s needed because otherwise sorting will happen only on the first digit of the number. For example, 1 and 10 will be considered as 1 because sorting will be done on the first digit. Hence, Terraform will consider 10 to put in front of 2 because it’s soring only using the first digit.

The custom key value is independent of the list index values. Hence, it won’t trigger the recreation of existing elements of the list in the event of any change to the list variable. Therefore, using this method, you can generate a custom key to achieve order preservation without recreating the existing elements of the list in the event of a list update.