Hacking First-Class Functions into Terraform HCL

Most developers who have written large amounts of Terraform HCL have felt its obvious lack first-class, user-defined functions. Let’s see if we can “fix” that.

User-defined Functions-ish

While Terraform’s documentation claims that Terraform doesn’t support user-defined functions, it actually does…kinda. If you squint a bit, you can pretend that modules are user-defined functions. After all, they have inputs, outputs, and you, the user, can define them. Here’s an example:

// In my_function/module.tf
variable a {
  type = string
}

output b {
  value = title(strrev(var.a))
}

// In main.tf
module call_my_function {
  source = "./my_function"
  a = "Hi there"
}

output out {
  value = module.call_my_function.b
}

As expected, this produces

out = "Ereht IH"

(See for yourself.)

So Terraform definitely allows us to define our own “functions.” They’re just very heavy-weight. Each function requires that you create a new directory and a new .tf file in that directory. Then calling that function requires at least 4 lines of code at every call-site (assuming you’re using normal formatting rules). So yes, it’s possible, but not very convenient.

First-class Functions

Inconvenience aside, however, there is one more thing about these “functions” that is extremely unfortunate. They are not first-class. In other words, they are not values that can be passed around between modules. Things that are first-class can be “picked up” and passed around. Terraform supports many first-class values, like numbers, strings, objects, etc. All of these can be passed into a module’s variables. But you cannot pass a module itself into another module’s variables.

That’s rather unfortunate, especially because most other languages these days support first-class functions. In JavaScript, for example, you can write

function my_function(a) {
  return a.split("").reverse().join("")
}

my_function("Hi there")
// 'ereht iH'

["Hi there", "folks"].map(my_function)
// ['ereht iH', 'sklof']

Notice how we passed the function my_function as an argument to .map. That’s because JavaScript treats functions as first-class values. Just like any value, functions can be stored in variables, passed as arguments, etc.

Is this kind of thing possible in Terraform? Well, as it turns out, it kind of is…

(ab)Using Terraform’s Features

One way to think of functions is as “delayed computation.” We want a way to describe an algorithm/transformation without actually computing it yet. But in Terraform, all variables are evaluated immediately (it doesn’t even short-circuit Boolean operators at the time of writing!). Is there a way to delay computation in Terraform without using a module? In fact, there is. And it’s probably not what you were expecting.

templatefile is an innocent-looking function in Terraform that actually does almost what we want. It lets you specify the path to a Terraform template in a file and then pass the arguments to the template as a object. For example,

// In template.tpl
Hello ${name}.

// In main.tf
output out {
  value = templatefile("${path.module}/template.tpl", {
    name = "Elliot"
  })
}

This produces Hello Elliot as you would expect.

templatefile with a template seems an awful lot like a function, but it only works for functions that produce strings. Could we abuse this to work for any Terraform value?

Why, yes. Yes we can:

// my_function.tpl
${jsonencode({
  b = title(strrev(a))
})}

// main.tf
locals {
  func_result = jsondecode(templatefile("${path.module}/my_function.tpl", {
    a = "Hi there"
  }))
}

output out {
  value = local.func_result.b
}

This produces the same output as the module version above:

out = "Ereht IH"

(See for yourself.)

How does it work? Since templatefile only works to produce strings, we can abuse the fact that Terraform also supports encoding and decoding strings as JSON. We can produce an arbitrary JSON value in our template and then immediately decode it after rendering the template. We can even see that Terraform treats this JSON-decoded value as a regular object!

$ terraform console
> type(local.func_result)
object({
    b: string,
})

We can now treat the paths to these “templates” as if they were function names. For example, I can pass this “function” to a module and replicate something like the JavaScript code we have above:

// In map/module.tf
variable items {
  type = list(any)
}

variable func {
  type = string
}

output out {
  value = [for a in var.items : jsondecode(templatefile(var.func, { a = a }))]
}

// In map_example.tf
module mapped {
  source = "./map"
  items = ["Hello there", "folks"]
  func = "${path.module}/my_function.tpl"
}

output map_example {
  value = module.mapped.out.*.b
}

(See for yourself.)

Recap

So what have we done? We’ve invented a new “calling convention” in Terraform using decodejson(templatefile(func, args)) where func is the path to a template that produces a JSON string. This calling convention supports first-class, user-defined functions even though Terraform doesn’t officially support them. As a side bonus, we actually made it possible to define these functions with only one file instead of requiring a new directory like modules do. And calling these functions can be done in any terraform expression. They don’t need to be defined as separate module blocks.

There are, however, some pretty major downsides to this idea as well.

First, it’s crazy. If you ever saw this in your Terraform code, you’d first be very confused and then probably very angry that someone pulled this trick in the first place.

Second, it’s obviously slow. We’re converting to and from JSON on every “function” call. If you have enormous objects it may not handle that well.

Lastly, this trick can only be used one layer deep. Yes, Terraform won’t let templatefile templates call templatefile. So that means you can’t do cool recursive things with these functions. Bummer.

Still, with all of that, we at least had fun. And, who knows, someone may find it useful too.