In this post, I will provide background information on using CI/CD with Terraform Enterprise, decision points to select the right Terraform Enterprise package, and finally a detailed walk through of an example Jenkins pipeline using Terraform Enterprise (TFE).
Terraform is the ubiquitous choice for infrastructure and cloud services provisioning. Organizations have built mature processes around the creation, review, and use of Terraform code to manage their infrastructure. One common integration point for Terraform is a CI/CD tool such as Bamboo, Circle CI, or Jenkins. The latter part of this post will focus on Jenkins, but other than pipeline code, the guidance is generally the same across other CI/CD tools.
CI/CD Interacting with TFE
The Tao of HashiCorp has 8 principles that are used to guide the company’s vision, roadmap, and product design. The first principle is Workflows, not technology as we focus on the end goal and workflow, not the underlying technology — because technology changes, but end goals stay the same. This workflow-first principle is very evident with TFE because you have the flexibility of using UI/VCS webhook, CLI, and REST API driven workflows.
How are you using Terraform with your CI/CD today? Are there established processes or controls that need to be respected when considering TFE? These questions and more are used to establish the adoption pattern for TFE relative to your existing pipeline.
The actual migration can be accomplished with minimal refactoring of your pipeline code. This is the approach that most organizations take when implementing TFE where the goal is to focus on extracting value from enterprise features without disrupting established workflows. For example, you have use cases being lead by security and governance for Sentinel Policy as Code and TFE Audit capabilities; the existing workflow and pipeline code should not need to be refactored completely though there will be some minor modifications.
TFE provides features to support the three main teams within IT often interacting with infrastructure and cloud services; Development, Operations, and Security. A sample mapping of enterprise features by team includes API access to Terraform for Development; Team Management, RBAC, and Multi-tenancy for Operations; and Sentinel Policy as Code for Security.
HashiCorp has two consumption models of Terraform that can be purchased with annual or multi-year subscriptions: SaaS and private installation. Recently the offerings were rebranded as Terraform Cloud (SaaS) and TFE (private installation). Terraform Cloud is a viable option for teams looking for collaboration features while TFE is most often deployed by enterprises with compliance, security, and/or scale requirements.
Which Terraform package is right for you? If you are just starting your journey with Infrastructure as Code (IaC) and have not encountered collaboration or governance requirements, then Open Source is probably fine for the next 6–12 months as you mature your IaC practice. But if you are in search of collaboration or governance features you should be evaluating Terraform Cloud or Terraform Enterprise.
TFE has a few hierarchy constructs that are relevant to your pipeline configuration. The top-level is an organization which can be used as a boundary between business units, environment stages, or other multi-tenancy criteria that may be important to your enterprise.
Within these organizations workspaces are where your Terraform code will be executed and will contain your state, variables, and policy sets will be applied. VCS integration and Sentinel policies defined at the organization level are applied to some or all of these workspaces. Finally, users are assigned to teams that are created within the organization to provide one or more levels of access. Team or user tokens are generated and are used to interact with TFE via the API.
OK, Enough Talking…
I’m going to present a basic Jenkins job file that will demonstrate a CLI-driven workflow by interacting with TFE using a Remote Backend file. The Remote Backend file tells Terraform OSS to use a TFE server to execute the Run remotely. Variables used for the Run and associated state file will reside within the TFE workspace as mentioned earlier. The files referenced in the walkthrough are in this repo TFE-Jenkins along with bootstrap scripts.
The next sections are a step-by-step guide to executing a Jenkins job against Terraform Enterprise. Assumptions are that you have a Jenkins instance running on Linux, Terraform Enterprise instance, access to a VCS repo, and around 15 minutes. I’ll start with an overview of the steps and then go through TFE and Jenkins configuration.
Overview of High-Level Steps
Create a Workspace in TFE
Set the Workspace to auto-apply (to allow Run to auto-apply if it passes Sentinel checks)
Specify the desired TF binary version for the Workspace
Set cloud credentials as Workspace variables (AWS in this example)
Create repo as source of the Workspace’s Terraform code
Seed the code repo with a remote-backend.tf
Configure .terraformrc file on server
TFE Workspace Prep
- Navigate in TFE: Workspaces > Jenkins Target Workspace > Settings > General
- Apply Method = Auto Apply
- Terraform Version = 0.11.14
- Save Settings
- Navigate in TFE: Workspaces > Select Jenkins Target Workspace > Variables
- Environment Variables
- Create the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY variables
- Use the “sensitive” feature of TFE to obfuscate the value in the UI
- Click Save Variable
Terraform Code Prep
- Store main.tf and backend.tf in a repo accessible to the Jenkins server, the repo URL will be specified in the job file as the GIT_REPO value
Jenkins OS Setup
- TFE .terraformrc file must be created and configured on Linux-based Jenkins server (see link for notes on configuring terraformrc for Windows)
- If not, you will receive “Error configuring the backend “remote”: required token could not be found” in the Stage Log
- sudo nano /var/lib/jenkins/.terraformrc
- Install unzip
- sudo apt install unzip -y
Jenkins Job File
- Update GIT_REPO value to reflect your target environment
- Example files are available at https://github.com/raygj/tfe-jenkins
Pipeline Setup from Jenkins UI
- http://< IP or DNS of your server>:8080
- New Item
- <some name>
- Definition = Pipeline Script
- Script = paste Jenkins file from previous section
- Leave all other items default/unchecked
Pipeline Run from Jenkins UI
- Jenkins > Build Dashboard
- Click on “clock” to schedule a run of your pipeline
- Click on the name to see the stages being executed
Success from Jenkins UI
Success from TFE UI
Using the components from this post you have what you need to interact with Terraform Enterprise using a CI/CD tool via Terraform OSS binary. A migration to Terraform Enterprise can be done while respecting your workflow and investments in accompanying pipeline components that may be north or southbound of Terraform.
The net result is that your organization will be positioned to take advantage of Terraform Enterprise features that could unlock new value or opportunities to increase the collaboration, governance, and stability of your infrastructure.