Terraform Patterns, Observed
Part 1: Module Types
This is purely my perspective as a practitioner with firsthand visibility into several working solutions in my career as a software consultant. Much of the vocabulary used in this series is of my own imagination and will surely cede to better nomenclature from the community. Moreover, many implementations I have seen in practice include multiple types of patterns discussed in this series.
This commentary is neither definitive nor endorsed and is in no way representative of the views of my employer, Accenture.
All Terraform code included in the execution of a plan is contained in at least one module. Oftentimes, a single plan includes multiple modules, especially those supporting large (or complex), automated implementations.
I will begin this series by presenting different types of Terraform modules to provide context for later narratives. I’ll also provide some general observations regarding the application of each module type. Finally, I’ll close this post with a comparison of two similar module types and when one might leverage one or the other.
Structural Module Types
These definitions focus mainly on the organization of module code, with respect to the portion of the state tree[1] starting at the module under review. Many of these types would be used in conjunction. They can be thought of as distinct shapes of building blocks that are designed to fit together in a specific way. I’ll start by introducing some terms used to describe the special properties of certain modules.
Structural Terms
Terminating — A module that references no other modules, thus terminating the Terraform state tree.
Unitary — A module with a single definition. Introduces a single state object[2].
Singleton
A terminating, unitary module with a single resource definition. In many cases, when these are also remote modules (defined below), this presents an extra, unnecessary node in the terraform state tree. In some cases, logic can be introduced to enforce standards or conventions[3]. Note that, like any module, these can contain any number of data definitions or any other blocks.
Bad for:
- Highly dependent resources (as they would have to be combined using module composition to be useful)
- Resources that have no unitary use case (i.e. they generally don’t exist on their own)
- Resources that might logically be considered a property of another resource
Good for:
- One-off solutions/experiments on singular platform resources
- Avoiding the mixing of resource blocks with module blocks in the same module
- The enforcement of standards via default variable values or hardcoded field assignments
Example — A VM-centric firewall resource solution operated by a single, centralized team over hundreds of instantiations, possibly driven by a service request ticketing system.
Terminating Composite
A terminating, non-unitary module with multiple resource definitions. In most cases, this is a good way to bundle the states of resources that are logically tightly coupled and that do not require a great deal of flexibility.
Bad for:
- Mutually independent resources (as resources in the same module will always share the same terraform state lifecycle [Note — this is true for all modules and could be asserted for all subsequent composite module types])
Good for:
- Very tightly-coupled resources that can be standardized for a common use case
Example — A solution that bundles a virtual private cloud resource and a standard set of subnetworks and that is intended to be reused at least a few times.
Nesting Composite
A non-terminating module that implements module composition on other modules local to the repository that are themselves either a singleton or terminating composite.[4] Note, this would be 1–1 with the module’s state object.
Bad for:
- Very tightly-coupled resources that depend on each other and are fit to a narrow set of use cases
- Introducing module composition on purely singleton modules
Good for:
- Less tightly coupled, or more flexible, resource bundles, where the individual submodules (defined below) may be directly useful in other solutions
- Establishing a preferred pattern for a particular solution, but expecting and enabling some deviation
- Modules intended to be referenced remotely by 3rd-party development teams
Example — A virtual machine solution with variability in boot disk, attached disk, and machine type, each of which is developed in its own distinct module, bundled together using module composition in a root module (defined below) that is preconfigured to support the most common use cases (and can be used as a guide to developing custom, uncommon configurations).
Abstracted Composite
A non-terminating module implementing module composition on multiple other (valid) remote modules.
Bad for:
- A single abstraction of other remote modules you also develop and that are not referenced elsewhere
- Composing co-dependent modules
Good for:
- Loosely coupled resource bundles that combine other complete solutions
Example — A “landing zone” or “foundation”[5] solution that is intended to establish hundreds or thousands of resources in a single, “big bang” operation
Heterogeneous Composite
A non-terminating, non-unitary module that mixes resource and module blocks (possibly in the same .tf file or even in different files, but within the same module).
Invalid — I strongly consider this an anti-pattern, more so than introducing a singleton module with module composition to avoid it[6].
Coiling Composite
A non-terminating module that references at least one local module (defined below) that in turn references a second local module[7]. I don’t consider this to be invalid, necessarily, but I do think it should be avoided. Similar to nesting composites, these are 1–1 with their respective state object.
Bad for:
- Most situations, as this is typically viewed as an anti-pattern
Good for:
- Large, complex state objects supported by code fully stored in a single repository.
Example — This might arise if you must adhere to some policy that e.g. requires all source code referenced in an automation pipeline to be stored in the same repository[8].
Wrapping
A non-terminating, unitary module that references a single other (valid) module. This is similar to singleton modules. It is generally recommended to minimize the use of wrapping modules, but they can be very useful to enforce Principal of Least Privilege and Separation of Concerns. Aside from hardcoding field assignments in the module blocks, the primary effort would be on variable and output definitions.
Bad for:
- A single abstraction of another remote module you also develop and that is not referenced elsewhere
Good for:
- Overriding default values in an active module referencing a 3rd-party remote module (both active and 3rd-party remote defined below)
- Enforcing standards or PoLP/SoC for internal use of such a module
Example — A virtual machine solution that references a 3rd-party remote module, thereby benefiting from the work of another team (and their development budget), but hardcodes certain values like operating system or boot disk size.
Functional Module Types
These definitions describe how a module is used (rather than how it is shaped).
Active
This is the target of a tf plan command and corresponds to the root of a state tree. It includes one or many .tfvars files (or equivalent[9], roughly corresponding to the module’s root-cardinality[10]). Often, the active module is also a root module, but this is not a technical requirement. Any structural type can be active.
Leaf/Leaf-most
This is a non-active module that has no module references (remote or local) and is located at a “leaf” node of a state tree. This may only be a singleton or terminating composite module and by definition will never be active.
Remote
This is a module that is referenced by another module using module composition and that is stored separately. This could also be an active module or it can be designed to be always referenced by another module configuration, but this will typically only be discussed in conjunction with another module (i.e. it won’t be the subject of discussion, so much as in relation to the subject). Any structural type can be referenced as a remote module.
Special Case — Inactive Remote
I’m including this only to identify it as a special case of remote modules. These are most commonly developed as Open Source Software (OSS) by members of the community. By design, they will never be an active module[11]. Instead, they are always intended to be referenced. Generally speaking, when discussing any remote module, it may be most convenient to consider it as an inactive remote. Thus, I will only make the distinction if and when it is illustrative to do so.
Special Case — 3rd Party Remote
These are always remote modules and are developed as OSS by members of the community. For our purposes, we will consider them exclusively inactive remote modules. Accordingly, we will use “3rd-party remote modules” as a synonym or synecdoche for “inactive remote modules”. Because this is a functional type, we’ll also generally consider these whenever it is appropriate to provide great flexibility or abstraction in a module across multiple use cases, even when we are the sole contributors to the 3rd-party remote module. The only drawback to 3rd-party remote modules is the likelihood to include complex logic in order to provide the level of flexibility it affords.
Positional Module Types
These definitions refer to purely positional (from the standpoint of a repository) aspects of a module and are used generically.
Root
This is the module at the root of a repository (if it exists). This is generally only worth discussing in the case of nesting composite (and the to-be-avoided coiling composite) modules but abstracting, wrapping, and singleton modules are each also commonly established as a root module[12]. A root module is a requirement of Hashicorp’s standard module structure.
Local
This generally refers to any non-root module. General style guidance (e.g. from Hashicorp re. Standard Module Structure) is to include these under individual subdirectories of the repository under the relative path of ‘./modules/’. Note that while standard style practices may be established to introduce a logically “private” module, any module stored in source control is accessible[13].
Remote, revisited
While I would consider a remote module to be a primarily functional designation, it is often referenced and considered in contrast to local modules from a positional standpoint. Indeed, this comparison is discussed directly in a subsequent post.
Submodule
Generally, this would be any module in the chain of references from the module under consideration. Typically, I will use this to refer to the direct references (i.e. a child relationship if module references were mapped out like a graph from the active module to the leaf-most), rather than any module in the chain.
Additional Thoughts
Terminating Composite vs. Nesting Composite
Terminating composite and nesting composite modules generally solve the same problem: combining resources that logically integrate into a bundled solution. The former forever ties the included resources together in the same state while the latter affords consuming teams to decompose the module, using it as a guide, and referencing only the necessary components.
Terminating composite modules have a high chance of violating DRY if sufficiently similar resources are needed in multiple places in different configurations. They can also complicate troubleshooting activities if the modules have several resources as a simple syntax error may become a needle in a haystack. Nesting composites have a high chance of introducing unnecessary nodes into the state tree, but like other composites, provide a more nuanced mapping of relationships.
A team intending to lock into a specific configuration (e.g. a compute instance with a single supported operating system and support for at most one attached disk) would likely gravitate toward a terminating composite module. A team intending to support more flexibility would likely develop a nesting composite module.
Next Time…
We’ll consider how various modules can be arranged to support full infrastructure installations.
Footnotes
[1] The state tree is defined as the representation of the live resources as recorded by a single `terraform apply` execution (more on this in a subsequent post).
[2] A state object is any sub-tree of a state tree (more on this in a subsequent post).
[3] It’s generally advised to minimize logic in Terraform code. More on this in a subsequent post.
[4] I believe this is what Hashicorp refers to as “composable”.
[5] Without going too far down this rabbit hole, here, this would be any combination of sufficiently different resource types that are not developed together but would be combined to establish a standard starting point for a given engineering team. Think the managed database, the storage buckets, the compute resource, and the API manager: you might develop them in their own repositories, each with their own versions and respective lifecycles, but want to compose them together for a typical LAMP-ish stack that you offer to your product engineering teams.
[6] Since the computation time of an automation pipeline is rarely the top KPI, and since this is capped in the most extreme case at doubling the total nodes in the state tree and adding at most 1 to the maximum state tree depth, I am erring on the side of readability and some semblance of “style”/code-cleanliness. I am very interested in a counter-argument.
[7] This creates deep state trees within the module and thus I believe it is what Hashicorp would consider “not composable”.
[8] Frankly, just avoid this pattern. If you choose to allow it as a pattern in your organization (outside of one-off solutions and experimentation), you should document your own specific justification.
[9] This could be some other input mechanism, but using a .tfvars file (or multiple) to store the configuration of a terraform state is the most common approach. Also, technically, the repository containing the “active module” code itself need not contain the e.g. .tfvars configuration (the code could be stored in one repository and the .tfvars in another), but we won’t consider advanced configurations that are deliberately designed.
[10] This concept probably warrants its own post and possibly warrants a better name, but this is the number of state objects that an active module acts upon. This may be best explained by contrasting a 1-root active module (where all configuration resides in e.g. a single .tfvars file that grows approximately linearly with N) and an N-root active module (where configuration is separated into N relatively uniform shaped .tfvars files). I believe all arities between 1 and N are possible, but I also believe there is no a priori way of slicing this that fits into a single-clause parenthetical.
[11] Unless the modules’ repository is forked and Frankenstein’ed.
[12] Or, rather, they could all be established at the root of a repository. In the cases of these latter three types, it is generally more useful to specify the type itself, rather than the positional aspect.
[13] Assuming one is authorized to access the module’s repo, in general.
