Testing Terraform with Terratest & GO

Published in

Credera Engineering

8 min readJun 20, 2022

Here, I’ll be going over how to test the Infrastructure As Code (IAC) of a previous project (Auto Tagger) with Terratest and Go.

Previously, we set up some IAC that would automate the process of applying tags to AWS resources. That’s cool and all, but wouldn’t it be nice if we didn’t have to confirm that the tags had been applied via the GUI?

Terraform 1.1.x will be used in parts of this tutorial, and so it is worth noting that some configuration may look different.

Please note that anything in angled brackets (<>) will need to be populated with values for your desired outcome.

Pre-requisites

An AWS account
AWS Auto Tagger
Terraform
Go version 1.18 and a basic understanding of it

Overview

The general idea is that we use Terratest to build our Auto Tagger, build a resource and check it has the expected tags, and then destroy our IAC.

We’ll structure the directories like this:

root
  |-- AWS_Auto_Tagger
  |    `-- all terraform files used to create the auto tagger IAC
  |
  |-- AWS_Auto_Tagger_Tests
      |
      |-- tf_test_resource_setup
      |   |
      |   |-- s3_bucket
      |        `-- s3_bucket.tf
      |
      |-- s3_bucket_test.go

We’ll go into more detail later, but in general, we’ll be pointing to directories and telling Terratest to apply the Terraform in those directories so we can test it.

Getting started

Make sure you have your directories structured as shown before. We’ll start off by creating a go.mod file by entering into the command line go mod init <root/repo>.com/AWS_Auto_Tagger_Tests (Here are notes on mod naming).

Set up the s3_bucket.tf:

# s3_bucket.tfterraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = ">= 4.4.0"
    }
  }
  required_version = ">= 0.14.9"
}resource "aws_s3_bucket" "test_bucket" {
  bucket = "auto-tag-test-bucket"
}output "bucket_name" {
  value = aws_s3_bucket.test_bucket.bucket
}

The output value is what we will use later on to retrieve the bucket and check to see if the tags have been applied.

Now, we’ll need to set up s3_bucket_test.go. Provide a package name and import the following:

# s3_bucket_test.gopackage testimport (
 "testing"
 "time"// This is Terratests AWS package, it contains a function that can 
// retrieve tags from a bucket.
 "github.com/gruntwork-io/terratest/modules/aws"// This is Terratests Terraform package, it contains a two functions // that we'll use to apply and destroy our terraform.
 "github.com/gruntwork-io/terratest/modules/terraform" "github.com/stretchr/testify/assert"
)

Run go mod tidy to add the import s you’ve just defined to the go.mod file. Here are a couple of links to the packages that we’ll be using:

Testing

Now that we’ve got the basics set-up, everything from here onwards is pretty straightforward.

We’ll define our test and add our Terraform setup:

# s3_bucket_test.go...func TestS3BucketTags(t *testing.T){ // retryable errors in terraform testing.
  autoTaggerDirectory := terraform.WithDefaultRetryableErrors(t, &terraform.Options{
    // This points to the directory where our auto tagger IAC is
    // located
    TerraformDir: "../AWS_Auto_Tagger",
  })
  // This will destroy the Terraform located in the variable you 
  // defined earlier
  defer terraform.Destroy(t, autoTaggerDirectory)
  // This will apply the Terraform located in the variable you 
  // defined earlier
  terraform.InitAndApply(t, autoTaggerDirectory)}

defer will run at the end of the test, regardless of the outcome. You must be careful when terminating tests with CTRL + C or CMD + C as this will stop the test before it can run the defer function, leaving your Terraform still applied in a live environment!

Now we want to do the same for our S3 Bucket:

# s3_bucket_test.gofunc TestS3BucketTags(t *testing.T){  ...S3Bucket := terraform.WithDefaultRetryableErrors(t, &terraform.Options{
    // Points to the directory containing our S3 Bucket Terraform
    TerraformDir: "./tf_test_resource_setup/s3_bucket",
  })  // This the same as before. It will apply our Terraform and then
  // destroy it after the test has finished running.  defer terraform.Destroy(t, S3Bucket)
  terraform.InitAndApply(t, S3Bucket)}

What we’ve done so far is apply the Terraform that will automatically apply tags to our resources and apply the Terraform that will create an S3 Bucket. Once the test has finished, you can destroy it all!

Now we can do a check to see if the tags have actually been applied and whether they match the expected outcome. We’ll be using the Assert package by Strechr:

# s3_bucket_test.gofunc TestS3BucketTags(t *testing.T){  ...  // Here we're defining what we expect our tags to look like
  user := "<name_of_your_user/role>"
  currDate := time.Now().Format("2006-01-02")
  
  // These are the variables that we will provide to the Terratest
  // AWS package to retrieve the tags from the bucket.
  region := "eu-west-1"  // This retrieves the value of the 'Output' value defined in our 
  // s3_bucket directory by providing the directory and the name of 
  // the 'Output' we want.
  bucketName := terraform.Output(t, S3Bucket, "bucket_name")  // Wait for the tags to be applied  
  time.Sleep(10 * time.Second)
  // Here we're using the AWS Terratest package to retrieve the 
  // bucket tags by providing the function the region and bucket 
  // name. This returns map[string]string
  actualBucketTags := aws.GetS3BucketTags(t, region, bucketName)  // And finally we test to if the tags are applied as expected
  assert.True(t, actualBucketTags["CreatedBy"] == user)
  assert.True(t, actualBucketTags["CreatedOn"] == currDate)}

I’m quite new to Go and haven’t figured out how to use a retry of some sort, so I use time.Sleep as a basic workaround. If you know how to get around this, feel free to use what you think works best.

Now that you’ve written the test, you can run go test s3_bucket_test.go in the command line. You should see something like this:

Alternative testing

If Terratest doesn’t provide a function to retrieve tags for a resource that you want to test eg. Elastic Cluster Service, you’ll need to set the tests up a little differently.

In the directory tf_test_resource_setup , define a new directory ecs_cluster and add the file ecs_cluster.tf to it. We’ll Terraform a simple ECS Cluster:

# ecs_cluster.tfterraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = ">= 4.4.0"
    }
  }
  required_version = ">= 0.14.9"
}resource "aws_ecs_cluster" "test_ecs_cluster" {
  name = "terratest-ecs-cluster"
}output "ecs_arn" {
  value = aws_ecs_cluster.test_ecs_cluster.arn
}

We’ll then add ecs_cluster_test.go to the AWS_Auto_Tagger_Tests directory and add the following:

# ecs_cluster_test.gopackage testimport (
 "fmt"
 "reflect"
 "testing"
 "time" "github.com/aws/aws-sdk-go/aws"
 "github.com/aws/aws-sdk-go/aws/session"
 "github.com/aws/aws-sdk-go/service/ecs" "github.com/gruntwork-io/terratest/modules/terraform"
 "github.com/stretchr/testify/assert" 
)

Instead of a simple Terratest package that we could import, we now need to work with the AWS SDK for Go. I’ll go over that more shortly but for now, we’ll continue setting up our test like before:

# ecs_cluster_test.go...func TestEcsClusterTags(t *testing.T){
  autoTaggerDirectory := terraform.WithDefaultRetryableErrors(t, &terraform.Options{
    TerraformDir: "../AWS_Auto_Tagger",
  })  defer terraform.Destroy(t, autoTaggerDirectory)
  terraform.InitAndApply(t, autoTaggerDirectory) EcsCluster := terraform.WithDefaultRetryableErrors(t, &terraform.Options{
   // Point to the directory containing your ECS cluster IAC
   TerraformDir: "../ecs_cluster",
 }) // Apply the ECS cluster and then destroy after test has run
 defer terraform.Destroy(t, EcsCluster)
 terraform.InitAndApply(t, EcsCluster)
}

So far it has mostly been the same — we’re just pointing to a different directory to set up the ECS cluster. We’ll now deviate a little and start using the new packages we imported earlier:

# ecs_cluster_test.go...func TestEcsClusterTags(t *testing.T){  ...  // We have to pre-define the values of the keys Key & Value and 
  // reference them in the Tag, otherwise it will throw an error. 
  user := "<name_of_your_user/role>"
  currDate := time.Now().Format("2006-01-02")
  createdBy := "CreatedBy"
  createdOn := "CreatedOn"  mockTags := ecs.ListTagsForResourceOutput{
    Tags: []*ecs.Tag{
      {Key: &createdBy, Value: &user},
      {Key: &createdOn, Value: &currDate},
    },
  }
}

Here, we’re defining the expected tags. When fetching the tags from ECS cluster, it will return a map the same as we have just defined. This is what we will compare the actual tags to.

Now, we’re going to retrieve the tags from the cluster. We need to create a session with the ECS client and provide the client the arn of the cluster we want:

# ecs_cluster_test.go...func TestEcsClusterTags(t *testing.T){
  
  ...  // Get the arn of the cluster from the output and create the input
  // that the ECS client will be able to understand.
  ecsArn := terraform.Output(t, EcsCluster, "ecs_arn")
  input := &ecs.ListTagsForResourceInput{
    ResourceArn: aws.String(ecsArn)
  }  // Create the ECS session 
  awsConfig := &aws.Config{Region: aws.String("eu-west-1")}
  svc := ecs.New(session.New(awsConfig))  time.Sleep(10 * time.Second)  // And combine the ECS session with the input to retrieve the tags
  ecsResponse, ecsErr := svc.ListTagsForResource(input)
  if ecsErr != nil {
    fmt.Printf("ERROR: %v\n", ecsErr)
  }}

svc.ListTagsForResources returns two values — the tags (if they exist) and an error (if one occurs). We must handle the error as it is defined (Go doesn’t compile if unused variables are present).

Finally, we can check if the tags are the same:

# ecs_cluster_test.go...func TestEcsClusterTags(t *testing.T){
  
  ...  // We only want the array of Tag maps
  actualEcsTags := ecsResponse.Tags
  expectedEcsTags := mockTags.Tags   assert.True(t, reflect.DeepEqual(actualECSTags, expectedEcsTags))
}

Because I’m new to Go, I haven’t figured out how to access items in an array as you would in JavaScript or Ruby, so I’ve gone for a cheeky workaround and made a deep comparison between the two objects.

Now, run go test ecs_cluster_test.go -v and you should see the test pass:

Notes

If you’ve already got the Auto Tagger Terraform applied, you need to remove the following from your tests:

autoTaggerDirectory := terraform.WithDefaultRetryableErrors(t, &terraform.Options{
    TerraformDir: "../AWS_Auto_Tagger",
  })defer terraform.Destroy(t, autoTaggerDirectory)
terraform.InitAndApply(t, autoTaggerDirectory)

Alternatively, run your tests in a different environment — if you don’t, this will destroy your Auto Tagger resources.

You should be able to test any Terraform with Go, Terratest, and the relevant cloud platform SDK. When to test is another question if you’re looking for a specific outcome to your IAC.

Conclusion

You’ve just finished testing the Auto Tagger IAC with Terratest. You could go a step further and implement the tests to run in a CI/CD pipeline to make sure everything is still running as expected.

I’ve taken quite a simple approach and I hope you’ve found this blog useful. Thanks to Credera for allowing me time to look into Teratest.

Here are some resources that I have found useful:

Interested in joining us?

Credera is currently hiring! View our open positions and apply here.

Got a question?

Please get in touch to speak to a member of our team.