Automating Multi-Environment Deployments with Terraform Cloud Workspaces

Intro

You probably are using/learning Terraform OSS (CLI) and you want to take advantage of the shiny features of Terraform Cloud (TFC), like automatic state file management, remote runs, securing variables, and many many others.

So how do you manage the complex relationship between workspaces and version control systems (VCS) repositories, deployment environments, and state files in Terraform Cloud? Especially if you are invested in Terraform OSS and have several state files and local workspaces. The concept of workspaces in Terraform Cloud is different from the concept in CLI — we will cover this in the next sections.

Terraform Cloud vs Terraform CLI

Terraform CLI is an infrastructure as code tool that lets you define and manage infrastructure resources through human-readable configuration files. It runs on your local machine and you will need it for this post.

Terraform Cloud is a SaaS-based platform that leverages the features of CLI and allows teams to work together, managing Terraform runs on a centralized, consistent and reliable environment instead of on your local machine, and many many other features. For a full comparison, please check this link

You can go through the example workflow on this post just using Terraform Cloud Free tier.

It All Starts From the VCS

You might be following the best practices on IAC by putting all your code in a VCS system, right? (Please say yes).

But the VCS strategy on how you are managing your infrastructure will define how much of a headache you will have.

Branching strategy really does not matter for Terraform Cloud, even if you are using subfolders in a single repo — but how you separate components is really the key here.

​​One of the reasons that Terraform configurations might become complicated and difficult to maintain is because of the monolith dilemma.

An example of a monolithic approach is when a large number of Terraform resources are defined in a single configuration. Resources can be accidentally destroyed by operators who shouldn’t have access to them. The operators only need to reference those resources, not control their existence or any of their attributes.

You know you need to refactor your monolith when:

• An unrelated individual can change/destroy infrastructure that they don’t need to own.
• Unneeded resources are defined and created only so that they can be referenced by other resources.
• A change to one resource triggers changes to other resources that didn’t need to change.
• Practitioners are not confident making changes to the infrastructure.

So when you are creating your repositories, think about:

• What resources do I need to create and control?
• What resources do I need to refer to when creating my own?

This conversation will lead to the next section: workspaces and state files.

What is a Workspace?

On OSS

In Terraform open source, a workspace is just an independent state file on the local disk.

For different environments, you could create different folders for each environment, with completely separate state data

But you can create parallel state data inside the same folder using the terraform workspace command (we will come back to this in a sec). Also using workspaces in Terraform OSS is not mandatory.

On Terraform Cloud

Terraform Cloud’s main unit of organization is a workspace. A workspace is basically a wrapper around a state file with a collection of:

• A VCS repo link
• Values for the variables
• Run history
• RBAC

and everything Terraform needs to manage this particular piece of infrastructure.

Why Workspaces Matter

Since one workspace is basically a state file for a particular environment + your organization permission structure — the recommended approach is to have one workspace per environment per set of infrastructure resources (or app, stack, whatever you want to define). Hence the conversation about the VCS strategy we had before.

For example, if you have a Web application in three environments (DEV, QA, PROD), you will have three workspaces. This way you can manage the three environments separately and deploy to one without affecting others. Each Terraform Cloud workspace will also have particular variables to its environment (including the environment cloud credentials).

(Why I think) Multiple folder strategy is not great

If having each environment put in a separate folder in the same repository sounds like a good idea at first, this will cause several issues:

• It’s difficult to maintain, as updates have to be carried at all folders.
• There could be disparities between configurations, so there won’t be a single source of truth. If something works in Dev, it might not work in Prod, and so on.
• VCS tools usually have a limit on the # of webhook connections that can be created per repository. Effectively creating a ceiling for a number of folders that can be used with this approach.

This is why having a single repo (single source of truth) with multiple workspaces is easy to maintain and we will see that with TFC RBAC, Sentinel and other features, we can mitigate a lot of risk of breaking stuff.

Is a Single Repo the Best Way?

Not necessarily. Terraform Cloud can adapt the VCS connection to branches and folders. It’s more about the RBAC on dividing workspaces.

The rule of thumb should be: One workspace per environment per terraform configuration, and division of workspaces and permissions should match your organisation’s division of responsibilities.

Coming back to the Monolith issue above: we can conclude that it is beneficial to break down the infrastructure into small manageable chunks, where component management and access make sense for your team structure and domains of expertise.

A quick example would be to manage network resources in a VCS repo that only the network team has access to, and this repo will be linked to one TFC workspace per environment (and in TFC only the network team will have access to apply/modify infrastructure).

The Cloud Block

In the past we had the remote backend configuration to specify which TFC workspace to use for the current working directory. The cloud block (TF 1.1.0 and above) includes an improved experience and more features.

terraform {
  cloud {
    organization = "andrepimentel"
    hostname = "app.terraform.io"
    workspaces {
      tags = ["hashicat", "aws"]
    }
  }
}

Take a look at the tags argument: this will allow you to change between local and remote workspaces. Also this will allow us to create Workspaces at Terraform Cloud with the desired tags. If you use the name argument instead, you will be able only to interact with a single workspace.

Let’s get technical: Building Workspaces from the ground up (CLI Workflow)

Requirements:

• A Terraform Cloud Account and Organisation
• Authenticate with Terraform Cloud — follow this tutorial
• Terraform CLI 1.10+
• Some Terraform code you want to bring into TFC. If you don’t have any, you can play around with the example repo here.

Let’s create the local workspaces if you don’t have any workspaces yet.

When running terraform init, Terraform will prompt you to input the name of your first workspace.

Let’s put hashicat-aws-dev as our first workspace.

And….magically the workspace appears at Terraform Cloud with the specific tags.

Now the next steps would be to configure your workspace variables and the VCS connection.

But let’s keep focusing on our CLI workflow for now.

Run the following commands to create two new workspaces:

terraform workspace new hashicat-aws-qa
terraform workspace new hashicat-aws-prod

And…BOOM! Your workspaces are now created and tagged properly in TFC.

If you run terraform workspace list you will see all your local workspaces and the current one you’re at.

To change workspaces, you can use the command:

terraform workspace select <name>

What about the state file?

This is already done :) you don’t have to worry about the state file anymore.

When you run terraform init with a remote backend or cloud block, Terraform detects your updated backend and confirms that you wish to migrate your state file to Terraform Cloud. It will upload the state file to the TFC Workspace.

Locally you will still have a state file, but the contents will pretty much say: It’s on TFC now.

For more details, please check the Migrate State to Terraform Cloud tutorial.

What If I Have 100s of Workspaces to Create?

Don’t panic! We got you covered with two solutions:

Using the TFC/E Provider (aka Terraformception)

You can use Terraform to manage TFC/E with this provider. Now you can:

• Manage any aspect from your TFC/E (users, teams, workspaces, policies, etc)
• Again, only TFC admins will have access to the VCS repo where this config resides, and they will also have the right RBAC on TFC.
• Consume assets in a self-service manner with modules.

Expanding a bit on the last: There is a very interesting module example here that allows you to create a list of workspaces with all arguments needed.

Using the API

If your to-go strategy is to create automation scripts for this task, there’s good news: you can use the Terraform Cloud API to do pretty much anything that’s available.

Action Point

What should I do next? What Should I do?

Take a look at our documentation around How to Evolve Your Provisioning Practices.

For advanced use cases and Terraform Cloud Team & Governance or Business tiers use cases, please check the Terraform Cloud Fundamentals tutorial (free).

Conclusion

Creating workspaces in Terraform Cloud is easy, the difficult bit is really defining what really should go into the workspaces and the right ownership. If you are deep into the monolith, very likely you will have issues at some point. But this is a chance to disentangle the code and enforce better collaboration and RBAC, it also might help clarify a cloudy (no pun intended) area of your infrastructure.