I’m going to guide you through how to quickly write and deploy terraform code using localstack and how to test your code using terratest. You can find the code on my github
Terraform is an infrastructure as code tool that lets you define resources in human-readable configuration files that you can version, reuse, and share. You can then use a consistent workflow to provision and manage all of your infrastructure throughout its lifecycle.
I’m a huge believer in the importance of testing and more over, I really like TDD (Test Driven Development) approach. When I started to use terraform it was fun, you iterate on your code, you try to deploy, you see it in your cloud console and you move forward like that. Unfortunately, soon enough I realized that this way of writting and deploying module/code is quite heavy and you loose a big amount of time. More often, I was testing the service in the console by hand since you can not be sure that what’s you deployed is working the way you expect it to work.
Based on this assumption, I started to look if I could find a better way to iterate over my code by having other tools to deploy my infrastructure faster and to have automatic testing and I found the combination of
Please, note that I provided examples on how to integrate localstack and terratest with gitlab-ci and github actions on my github
Terratest is a Go library that makes it easier to write automated tests for your infrastructure code. It provides a variety of helper functions for common infrastructure testing tasks like testing terraform code and deploying it but it provides a lot more.
Localstack provides an easy-to-use test/mocking framework for developing Cloud applications. It spins up a testing environment on your local machine that provides the same functionality and APIs as the real AWS cloud environment like AWS. You can use localstack in your CICD of locally.
How to deploy locally with terraform and localstack.
For this article we’re going to use docker-compose.
After creating your docker-compose.yml file, execute docker-compose up on a terminal to start localstack. Now you can deploy resources locally but first you need to setup your
provider.tf to use the localstack endpoints with terraform and not the AWS one. Here is an example.
In this provider, we still need to specify mock credentials for the AWS with access and secret key. To avoid issues with routing and authentication we add those four parameters
skip_requesting_account_id . All the parameters are speak for themselves except maybe s3_force_path_style. This one specify whether to enable the request to use path-style addressing, i.e.,
https://s3.amazonaws.com/BUCKET/KEY. By default, the S3 client will use virtual hosted bucket addressing,
https://BUCKET.s3.amazonaws.com/KEY, when possible. Specific to the Amazon S3 service. https://registry.terraform.io/providers/hashicorp/aws/latest/docs#s3_use_path_style
With this provider, you are now able to deploy resouces locally ! You need to specify every endpoints you’re going to use within your deployment. Here we only put the one we need.
All tests using Terratest are written in Golang. It is pretty convinient because as you know, terraform is developped using go, same as most of the DevOps tools.
Here is a example to show you how easy it is to test your devlopment. This piece of code, will run a
terraform apply --auto-approve, retrieve the exposed
outputs, assert them and
terraform destroy --auto-approve.
As you can see, in the
terraformOptions, we specify the directory where we wrote the instanciation of our terraform module for testing purposes. We also provide the file name to feed variables values. You have a lot of other options like
After, you need to define the
terraform destroy action. By giving the
defer keyword, you’re instructing this to destroy the resources created by the test after the
apply and any additional testing are done regardless of whether the test passes or fails.
Last but not least, you’re actually running your
terraform init and
terraform apply. If either the initialization or
apply fails, the test is considered a failure. Otherwise, the test is a success.
Well, now that we have our setup, we need to do some testing otherwise it’s not enough!
I added two assertions both on the
ID and the
ARN that our module outputs. Before that, the function wasn’t testing anything besides that the resources were created. But actually, even without tests, it’s very useful to be able to test your module localy. It allows faster development in my opinion if you can deploy your infrastructure localy.
For the sake of this article, I created a very simple terraform aws s3 bucket module to easily activate the versioning. As you can see, for each buckets, I test both outputs : the
ID and the
ARN of my bucket.
You can find the full code on my github
More testing with golang.
At this point, I was quite excited by the possibilities of having terratest combine with localstack. So what about more testing to verify if the versinioning is well activated and some functional testing to put, get, delete and assert the put value with the retrieved value from the get.
Test the versioning is well activated
First in order to execute more test, I created a function not to duplicate the declaration of the
For this test and the next ones, I will need a s3 session from aws gloang sdk in order to perform api calls on the deploy bucket in localstack environment. For that, I declared a global variable with an init function which gives me a reusable s3 session.
You can see two unusual inputs in the instanciation of the s3 session. Both
s3ForcePAthStyle are important here.
s3ForcePAthStyle tells the sdk to use the url
hostname/bucket instead of
bucket.hostname which allows the sdk to query the localstack bucket. Without those two inputs, the sdk will always try too hit AWS.
Once this setup is done, there is not much left to do. We just need to create a function to retrieve the value of the bucket versioning and to assert the returned value to verify the versioning has been activated.
If you’re still with me, I’ll show you how to test that you have a functional bucket with a put, a get, a delete and an assert of the value before the put annd after the get.
Like we iniated a s3 session for the previous test, we do not need to do it again.
Once this is done, we just need to extract the content of the file.
Don’t forget to delete the file you put, otherwise the terraform delete will fail because the bucket is not empty.
One last thing I’d like to introduce you is
gotestsum . This tool execute a
go test command but gives you a more readable output like
You can find all my code on github.
I also provide examples on how to integrate localstack and terratest with gitlab-ci, github actions and circle-ci.
Thanks for reading,
Don’t hesitate to post your thoughts in the comment section !