Pre-Commit Hooks for Terraform

Enforcing Terraform Linting and Security Standards Using Pre-Commit Hooks

Published in

Slalom Build

7 min readNov 20, 2020

Terraform has various independent native and open-source tools that check for code smells, enforce policies, and identify common security flaws. These tools help create consistent and curated Infrastructure as Code when used in conjunction with a sprinkling of automation. We can use Pre-Commit Hooks as an on-demand tool to integrate and use these tools before checking code into source control.

The Tools

Terraform Lint

TFLint is a linter that checks for possible errors, best practices, etc. It will also help identify provider-specific issues before errors occur during a Terraform run. For example, if Terraform is creating an EC2 instance and a developer accidentally references an invalid instance type, TFLint would flag it as an error.

TFSec

TFSec uses static analysis of your Terraform templates to spot potential security issues. TFSec checks for sensitive data inclusion across all providers and for violations of AWS, Azure, and GCP security best practice recommendations based on specific rulesets. For example, you would see an error similar to this if Terraform is creating a security group that is open to the world:

AWS006An ingress security group rule allows traffic from /0.Explanation:
Opening up ports to the public internet is generally to be avoided. You should restrict access to IP addresses or ranges that explicitly require it where possible.

Checkov

Checkov is a static code analysis tool used for infrastructure-as-code. It has wide ranging use-cases like Terraform, Terraform plan, Cloudformation, Kubernetes, Dockerfile, Serverless or ARM Templates. In the case of its usage in the configuration of the pre-commit hook, it will be used as a tool to cover generic Terraform security and compliance. Checkov can also regulate the creation, management, and updates of IaaS, PaaS or SaaS managed through Terraform. As an example, this is a type of error you would get if an S3 bucket is configured incorrectly:

Passed Checks: 1, Failed Checks: 1, Suppressed Checks: 0
Check: "Ensure all data stored in the S3 bucket is securely encrypted at rest"
/main.tf:
  Passed for resource: aws_s3_bucket.template_bucket 
Check: "Ensure all data stored in the S3 bucket is securely encrypted at rest"
/../regionStack/main.tf:
  Failed for resource: aws_s3_bucket.sls_deployment_bucket_name

Terraform Docs

Terraform Docs is a utility to automatically generate documentation from Terraform modules and base repositories in various output formats. Where this tool shines is in conjunction with the Pre-Commit hook by auto-populating the output of the tool into a repo’s README file which is described in detail later in the article.

Terraform Fmt

Terraform Fmt is a Terraform command that is available natively that is used to rewrite Terraform configuration files to a canonical format and style. This command applies a subset of the Terraform language style conventions, along with other minor adjustments for readability.

As an example, this would make sure that a resource has a consistent syntax for readability.

Before Terraform Fmt:

resource "aws_s3_bucket" "bucket" {
  bucket = "my-tf-test-bucket"
  acl = "private"  
  tags = {
    Name = "My bucket"
    Environment = "Dev"
  }
}

After Terraform Fmt:

resource "aws_s3_bucket" "bucket" {
  bucket = "my-tf-test-bucket"
  acl    = "private"  
  tags = {
    Name        = "My bucket"
    Environment = "Dev"
  }
}

Terraform Validate

Terraform Validate is “a native Terraform command that validates the configuration files in a directory, referring only to the configuration and not accessing any remote services such as remote state, provider APIs, etc.”

Terraform Validate can be used as a process dry-run and will check for any glaring flaws before a Terraform run. For example, If you have a variable that is currently unavailable or incorrectly referenced, Terraform validate will catch that error.

Error: Reference to undeclared input variable  on provider.tf line 2, in provider "aws":
   2:   region = var.regioAn input variable with the name "regio" has not been declared. Did you mean
"region"?

Setting Up the Pre-Commit Hook

Now that we know the available tooling independent of each other, let’s see what this looks like in practice. For our example, we will be using a Terraform repo with a file structure as shown below as well as creating.pre-commit-config.yaml and .tflint.hcl:

.
├── Makefile
├── README.md
├── main.tf
├── outputs.tf
├── provider.tf
├── .pre-commit-config.yaml #Will create this
├── .tflint.hcl #Will create this
├── requirements.tf
└── variables.tf

In order to use Pre-Commit with these tools:

The Terraform code resides in a git initialized directory.
There must be a localized .pre-commit-config.yaml that contains configuration for the pre-commit utility.
An optional rules list in a file called .tflint.hcl to enable enforcing tags on AWS resources.

Initial Setup

To install all of the Terraform tools using brew we will run the following commands from a Macbook with Brew installed:

brew tap liamg/tfsec
brew install terraform-docs tflint tfsec checkov
brew install pre-commit gawk coreutils

Add the Example Pre-commit Config File

Here is an example .pre-commit-config.yaml that we will use:

repos:
- repo: https://github.com/gruntwork-io/pre-commit
  rev: <VERSION> # Get the latest from: https://github.com/gruntwork-io/pre-commit/releases
  hooks:
    - id: tflint
      args:
        - "--module"
        - "--deep"
        - "--config=.tflint.hcl"
    - id: terraform-validate
    - id: terraform-fmt
- repo: git://github.com/antonbabenko/pre-commit-terraform
  rev: <VERSION> # Get the latest from: https://github.com/antonbabenko/pre-commit-terraform/releases
  hooks:
    - id: terraform_tfsec
    - id: terraform_docs
    - id: checkov

We also set specific arguments to use TFlint. The --module flag is extremely useful for a Terraform repo that contains a module by running a check based on the input variables passed to a module. The --module flag invokes a provider’s API to do a more detailed inspection. For example, it can run a check against an IAM profile name defined somewhere in your code as a hard-coded value. If the repo does not contain any modules, the argument is omitted.

In the yaml file, notice how we set rev:to<VERSION>. After creating the file, the pre-commit autoupdate command will pull the latest release from the respective git repositories by its most recent git tag and populate that into the <VERSION> flag. Keeping the version pinned is a great recommendation but incremental updates are also fine based on new features and releases to each pre-commit repo.

Add the .tflint File

TFLint also allows some custom rules to be defined by a specific ruleset. One rule that we can use is enforcing tags on resources based on required keys which we will put into our .tflint.hcl.

rule "aws_resource_missing_tags" {
  enabled = true
  tags = [
    "Environment",
    "Region",
    "Version",
  ]
}

This rule only checks for the tagging keys rather than the value. This will also not check if tags are being set as a variable since those are not available to the linter at runtime and only populated to the resources on a Terraform run.

Updating the README

To allow terraform-docs to auto-populate values into the repo’s README, we will add in these two lines:

#### Table of Contents
1. [Usage](#usage)
2. [Requirements](#requirements)
3. [Providers](#Providers)
4. [Inputs](#inputs)
5. [Outputs](#outputs)## Usage*various commands<!-- BEGINNING OF PRE-COMMIT-TERRAFORM DOCS HOOK -->
*auto populated information<!-- END OF PRE-COMMIT-TERRAFORM DOCS HOOK -->

Here is an example of a pre-commit run with auto-populated values from a README file:

<!-- BEGINNING OF PRE-COMMIT-TERRAFORM DOCS HOOK -->
## Requirements| Name | Version |
|------|---------|
| terraform | ~> 0.12.0 |
| aws | ~> 2.0 |## Providers| Name | Version |
|------|---------|
| aws | ~> 2.0 |## Inputs| Name | Description | Type | Default | Required |
|------|-------------|------|---------|:--------:|
| account\_name | account name | `string` | `"dev"` | no |
| deletion\_window | key deletion window in days | `number` | `30` | no |
| key\_description | kms key description | `string` | `"This create a kms key for dev"` | no |
| key\_purpose | the purpose of this key. The alias created by combining account\_name-key-purpose | `string` | n/a | yes |
| key\_rotation | enable key rotation | `bool` | `true` | no |
| kms\_policy | key policy | `string` | `null` | no |
| tags | Default Tags | `map` | n/a | yes |## Outputs| Name | Description |
|------|-------------|
| kms\_key\_alias | n/a |
| kms\_key\_arn | n/a |
| kms\_key\_id | n/a |
<!-- END OF PRE-COMMIT-TERRAFORM DOCS HOOK -->

Running the Pre-Commit Hook

Once we have everything set up we can then either install the pre-commit hook or run it manually. For this example, we will be running the pre-commit hook manually.

pre-commit run -a

When running this command from the root of our Terraform folder we may get an output similar to this:

pre-commit run -a
tflint........................................................Passed
Terraform validate............................................Passed
Terraform fmt.................................................Failed
- hook id: terraform-fmt
- files were modified by this hookmain.tfTerraform validate with tfsec.........................................................Passed
Terraform docs................................................Passed

We can see that terraform fmt fixed a syntax issue relating to Terraform code that was within the main.tf file.

Terraform docs will usually “fail” on a first run. The reported failure reports back a change within the README based on changes or additions to the populated values as they were pulled from the Terraform repo.

Closing Thoughts

By using Pre-Commit hooks, we can stitch together the various open-source and Terraform native tooling in an automated fashion using a single tool. This reduces the burden of downstream CI by putting the onus on the enforcement provided by Pre-Commit hooks. An additional benefit of using this process is quick identification of small issues upon each commit, which allows for cleaner pull requests and less review time.

Overall, Pre-Commit hooks go a long way toward increasing confidence in Terraform code that is submitted to a repo, reaping invaluable improvement in both developer proficiency and code quality.