Terraform?

Terraform by HashiCorp

At Glasswall our SRE (Site Reliability Engineering) team currently manages our IaC (Infrastructure As Code) deployments via Azure CLI. This is great for having direct access to the latest Azure resources however we are coming up against some challenges we’d like to solve:

• Our IaC is tightly coupled to Azure as its Azure CLI. If we wanted to move to AWS we would have to learn a new toolset.
• It can be brittle. Microsoft regularly changes their CLI which can generate a lot of maintenance work and frustration.
• Testing can be difficult. If you want to test the IaC you need to be able to quickly build, test and then tear down. This isn’t built into Azure CLI.
• It’s difficult to visualise what is going to be built or changed. If you have a large set of Azure CLI commands unless you have a full understanding of what is currently deployed its difficult to know the impact of your changes.

So how can Terraform solve these challenges for us?

What is Terraform?

Before understanding how Terraform can help us, what actually is Terraform?

As the name suggests it’s for building and changing the environments it is interacting with. It allows you to use a high-level configuration language. For example, creating a Redis cache in Azure is simple as:

Azure Redis Cache definition.

This simplicity and simple to use configuration language have helped make Terraform one of the most popular IaC tools within the industry.

Cloud agnostic, sort of.

In an ideal world, anyone that maintains IaC would love to be able to deploy into Azure, AWS or DigitalOcean with a single code base but as they all offer services in different formats this can be difficult. With Terraform we can hit a middle ground: a common configuration language.

With Terraform we can use a common configuration language to interact with different cloud providers.

Single technology, multiple clouds.

This common configuration language allows teams to deep dive into one particular language and build specialties. This, of course, doesn’t remove the learning curve with each new cloud provider we support but it takes the initial edge off.

Abstracting the brittle.

Another benefit of using a single configuration language is it abstracts away the use of the cloud SDK directly. The challenge of the constantly changing CLI and strange nuances are all managed by Terraform. We no longer have to manage the changes, HashiCorp does.

Testing

As with standard Software Engineering, SRE engineers have to make sure what they are deploying or managing has been tested. How do we know what we are going to deploy is correct and what we designed? Gruntworks has us covered.

Gruntworks has an amazing opensource Go testing library for testing Terraform code called Terratest.

https://github.com/gruntwork-io/terratest Usage example.

From this simple example we can answer the basic questions:

• Is the configuration valid syntax?
• Are there any mistakes in the use of Terraform?
• Will it create what we asked for?
• Are the services up and running?

Gruntworks is able to do this amazing work because of the power of Terraform's quick provisioning and down tearing. In any testing framework, you need to be able to quickly set up the environment and then, in turn, tear it down quickly to run the next test.

Visualise the actions

As with any action we perform on our infrastructure we want to know what our code change will actually change.

https://github.com/coinbase/terraform-landscape AWS Terraform Plan example

The above example shows what is being added, green, what’s being modified, in yellow and anything that is being deleted is in red. When developing or modifying infrastructure visualisations like this are invaluable to engineers in helping them understand what impact their change is going to have.

The way Terraform does this is by storing the state of the infrastructure in a remote store and in this case, it is Azure Blob Storage. Terraform can then know what is currently there and how it will be affected by the code change. Some see this as a downside by having to store a separate remote store but for us, this is invaluable in planning infrastructure changes.

Not all perfect

As with any technology, there is no “silver bullet” to solve all of our challenges and one of Terraforms strengths is also its weakness. The abstracted configuration language. So what are the downsides of this?

• If any new infrastructure from a provider comes out we have to wait for HashiCorp to update their libraries for us to use it.
• Some of our networking infrastructure is not fully implementable with Terraform.

To solve this we will have to use a hybrid approach of Azure/Cloud Platform CLI/SDK and Terraform to fill the gap until these challenges are solved.

In the end, the benefits for us completely outweigh the downsides of Terraform. Keep an eye on GlasswallEngineering as we transition over to Terraform.