Industrialize infrastructure as code in Government and Industry
Terraform agents’ best practices for regulated industries
By: Joshua Tracey, Resident Engineer, HashiCorp Federal, & Greg Thomas, Sr Solutions Engineer, HashiCorp Federal
Recently, an interesting post by Defense Scoop announced that “Intelligence agencies (are) confronting challenges with multi-cloud environments.”
Modern information technology (IT) abstracts infrastructure from physical hardware using logical constructs programmed in software. The challenge is having a unified way to provision infrastructure and take advantage of multi-cloud despite the skills gap, cost, compliance security, and operational overhead.
Most major Cloud Service Providers (CSPs) have integrated IaC tools within their vertical platform, such as Amazon Web Services Cloud Formation or Microsoft Azure Resource Manager. However, these tools do not scale horizontally across multiple CSPs, support traditional datacenters, or support bare metal. Terraform provides IT administrators with the horizontal, multi-cloud workflow they need.
In our last article, infrastructure as code in government and regulated industries, we learned how to use a platform approach versus DIY scaffolding to obtain value from IaC in regulated industries and government. To effectively take advantage of multi-cloud in regulated industries and government, a platform approach is needed to close the skills gap and tighten security.
This article reveals solutions that did not exist until recently. We aim to explain the often-overlooked benefits of Terraform agents and Terraform integration with Service Now, taking a platform approach to IaC for multi-cloud environments.
Challenge
Behind “push button automation,” there is always code. Even though we can abstract it away, it has to be codified. Terraform Cloud or Terraform Enterprise (TFC/TFE) are not designed to supplement CI/CD platforms. In most cases, work must be done before and after a Terraform plan and apply, or sometimes between a plan and apply. This leads to creating complex pipelines that, at some point, kick off a plan and apply against a TFC/TFE workspace and then have a mechanism to report back to the pipeline when the plan or apply is finished.
Solution
The hooks for agents help alleviate this by allowing actions pre-plan, post-plan, pre-apply, and post-apply. Imagine you wanted to run a pre-plan action on your Terraform code, then a post-plan action to validate the plan has applied, rather than waiting for a Terraform apply to fail 15 minutes in. Imagine you also wanted to send an API call somewhere to notify someone after that apply has finished.
You can take advantage of countless options and use cases like this with Terraform agents. You can also increase security by having a clear delineation point between agent pools assigned to respective workspaces.
Examples
Imagine an agency has a custom API to determine if resources can be provisioned. When everything is ok, the API returns “200”, meaning everything is ok, and resources are permitted to be deployed. In an emergency, audit, or security issue, the agency can push a button and force this API to return a 403 not-permitted. This gives complete control over all the infrastructure resources or assets deployed at a very granular level.
Below is an example of a custom Terraform Cloud agent configured with a pre-plan hook that checks the API. If the API returns status 200, the pre-plan hook exits with a 0 exit code, allowing the plan to continue.
Next, a post-plan hook runs. In this particular deployment, an Azure Storage Account is being created. The name is always dynamically generated, so the team wants to ensure the generated name is globally unique before moving to the apply phase, waiting a few minutes, and then finding out the name is already taken. In this case, it saves time by allowing us to fail fast, iterate on the code, and deliver the mission faster. Over the hundreds of thousands of resources deployed, this can save upwards of 10K hours a year.
Because all workspace variables are passed to the Terraform Cloud agent as environment variables, we can securely interpolate variables into the script that checks the Azure API for the storage account name. The $AZURE_ACCESS_TOKEN variable in the script is taken from the Terraform Cloud workspace variable below. The same applies to the $ARM_SUBSCRIPTION_ID and $STORAGE_ACCOUNT_NAME variables.
Finally, a post-apply hook is run. In this example, an internal system needs to be updated for audit purposes, a notification is sent via webhook to the workspace owner, and the Ansible automation platform API is called to trigger additional configurations.
Challenge
Resources often must be deployed into multiple secure and isolated environments. These environments usually cannot allow inbound connections from Terraform Cloud or Terraform Enterprise. Some of these environments only allow communication to cloud services via private endpoints. Other environments, such as air-gapped, only allow communication to internal applications such as vSphere from within the network. Note that there is an air-gapped Terraform Enterprise installer.
Solution
Terraform Cloud agents can be deployed into any isolated network to help solve this problem. Agents only require outbound connectivity to Terraform Cloud or Terraform Enterprise. The agent architecture is pull-based, so no inbound connectivity is required. Any agent you provision will poll Terraform Cloud or Terraform Enterprise for work and carry out the execution of that work locally.
You can leverage ServiceNow and Terraform Enterprise for “push button automation” to get the most value out of multi-cloud environments. This approach allows you to follow the best practices of infrastructure as code within an enterprise for compliance and security, cost savings, and isolation for billing and programmatic purposes. By connecting ServiceNow to Terraform Cloud or Terraform Enterprise, the Terraform Service Catalog integration enables platform teams to provision self-serve infrastructure by creating workspaces and performing Terraform runs from the ServiceNow interface.
Thank you for reading and learning more about HashiCorp’s infrastructure automation solutions. You can also check out Josh’s YouTube channel, The Sudo, and follow me on LinkedIn for more similar content.
