Manage Heroku Infrastructure with Terraform

Thomas Nyambati
Sep 3, 2018 · 11 min read

Heroku is the defacto Platform as a Service for many Developers. This popularity is based on the simplicity of provisioning and fast deployment cycles. As much as the Heroku team have done an admirable job of abstracting most of the complexities, managing the Heroku apps can be a little tedious as all the provisioning has to be done via the dashboard. When managing many apps, this approach might be slow and time-consuming.

With the advent of agile, the way we build and ship apps and systems has changed. System components can be built and shipped faster than before, but this has made managing and testing all the builds manually a nightmare. To be at par with the ever-changing industry, several methodologies have been adopted, to make the process more subtle, simple and efficient — IaC


Infrastructure as Code

According to Wikipedia, Infrastructure as Code (IAC) is a type of IT infrastructure that operations teams can automatically manage and provision programmatically, rather than using a manual process.

The concept of IaC is similar to programming scripts, which are used to automate important IT processes. However, scripts are primarily used to automate a series of static steps that must be repeated numerous times across multiple servers. Instead of scripts, IAC uses higher-level or descriptive language to code more versatile and adaptive provisioning and deployment processes.

Based on our use case Heroku, we can describe how our infrastructure should look like in code. This gives us the flexibility of updating, modifying or destroying it with few commands and time.
Below are some of the tools used in IaC.

  • Ansible
  • Terraform
  • Chef
  • Puppet

In this article we will be focusing on Heroku and Terraform.


Terraform

Terraform is an open source tool that codifies APIs into declarative configuration files that can be shared amongst team members, treated as code, edited, reviewed, and versioned. Terraform enables you to safely and predictably create, change, and improve your infrastructure.

Apart from the simplicity of Terraform, this tool can be used with different providers making it the best choice when working with different service. It also embraces the separation of concern (modularity), which makes it easy to break down the infrastructure into small components that can be re-used and easily maintained. You can find more details on Terraform on the official website.

Heroku and Terraform

Terraform works seamlessly with Heroku API. It simplifies how we interact with Heroku, making provisioning of Heroku infrastructure as simple as running terraform apply. Through its declarative language HCL, we can express the resources we need and terraform will take care of creating them for us.


What we will be building

In this article, we will be building a simple Heroku infrastructure as a demonstration. Below are the services we will be provisioning.

  • Pipeline
  • 3 apps (develop, staging, production)
  • 3 databases(add-ons)

Pre-requisites and Installation

Before we proceed, ensure you have a Heroku account and terraform binaries installed on your machine. Terraform binaries can be downloaded from the website. There is support for MacOs, Linux, FreeBSD, OpenBSD Solaris, and windows. For MacOs users you can install terraform via brew package manager.

We would be using Heroku throughout the article, therefore if you don’t have an account ensure you signup before we proceed.


Setting up the project

At this point, you should have a Heroku account set up, and terraform binaries installed on your machine. To confirm that terraform has been correctly installed, run terraform version from the command line. You should get a response similar to the one below.

The version may be different from mine depending on what version you have installed, nevertheless, the functionality will still be the same. Run the command below to create a working directory and initial terraform files.

Terraform files have an extension of .tf with and exception of .tfvars files that hold variable values. You can refer .tfvarsfile as .env for terraform and therefore should not be versioned at all unless they do not contain sensitive credentials that might compromise your infrastructure.

After running the commands your working directory should look as shown below.

Full code for this article can be found on our GitHub repository. Feel free to tinker with it as you see fit.

Setting up Heroku Provider

Terraform is agnostic to the underlying platforms by supporting providers. A provider is responsible for communication with the API and exposing resources. Terraform supports a lot of providers which can be found in the providers’ section of the documentation.

In our case, we will be using the heroku provider which will be responsible for communicating our intent to through the Heroku API. Heroku provider docs can be found at the Heroku section under providers.

The snippet below shows the syntax of creating a Heroku provider, and it to the main.tf file.

Providers are defined using the providerkeyword followed by the name of the provider in this case heroku. The Heroku provider requires two arguments email and api_key in order to communicate with your Heroku account.

We have used variable interpolation to dynamically pass this values to the provider. This not only makes our scripts dynamic it enables us to not expose sensitive values to the world. All variables used must be declared prior to being used. For more information refer to terraform variables documentation

The snippet below shows variable declaration syntax.

All the above arguments are optional, however, when the variable type is not specified, terraform will assume string type by default.

Before you can use terraform you need to run terraform init. This command will look at our configuration and download and install all the plugins required by Terraform.


Create Heroku Pipeline

Heroku pipelines allow us to group our application environments. This means you can have your dev, staging and production apps grouped in the same place. This setup provides a much easier way to manage your applications, especially if you have many applications in one account.

Traditionally you will have to navigate through the Heroku dashboard to create a pipeline, which might take 2–3 minutes. But with terraform, it gets easier.

All terraform resources assume the following syntax.

Add the code snippet below to pipeline.tf

Update the variables.tf file with the newly added pipeline variable.

The code above instructs terraform to create a Heroku pipeline identified as a pipeline. The identifier here is arbitrary, you can name it anything you want. This resource takes one argument name which is used to identify the pipeline created on Heroku.

Note: All variables declaration will be done in variables.tf file. This makes it easier for us to track and change the variables when the project scales.


Create Heroku Apps

Now that we have our pipeline setup, let's create apps that will be attached to this pipeline. Terraform creates Heroku apps via heroku_app resource. This resource takes only one required argument name , however, you can pass other arguments to customize your app further if required.

Note: All terraform resource take arguments, these arguments can be found under Argument Reference on each resource documentation.

Add the code snippet below to your apps.tf file.

Update the variables file with the newly added variables

Let's talk about what the code we have written above does. In apps.tf we have declared three apps resources identified as heroku_develop_app, heroku_staging_app, heroku_production_app. Terraform will create three Heroku apps when this code block is executed.

Note: Its Important at all times to give your resources unique names to avoid conflicts and also make it easier for terraform to track your infrastructure changes.


Attach Database Add-ons to the apps

Having setup up our apps, we need to create database add-ons that our deployed apps will use to persist data. To create a Heroku add-ons via Terraform we will use the heroku_addon resource. This resource takes three arguments app, plan and config . More information can be found on the Argument Reference section in the documentation.

Add-ons syntax

In our case, we are adding a Postgres database basic plan heroku-postgresql:hobby-basic. Add-ons plans can be found on each Heroku addon documentation.

Update the variables file with the snippet below.

The code above creates three database add-ons and attaches them to the three apps we created earlier.


Attach Heroku Apps to Heroku Pipeline

Now that we have our apps, the only thing left is to attach them to the pipeline we created earlier. This is made possible by the heroku_pipeline_coupling resource. Once the apps have been created, they can be added to different stages in the pipeline.

One thing you will notice that the app and pipeline arguments have been interpolated by variables we have not defined. This is because these values are outputs of other resources. To reference the value of another resource, you have to follow the following syntax.

Using develop example, we have referenced develop app name as heroku_app.develop.name, heroku_app being resource name, develop the unique identifier we gave the app earlier and the name which is the output value.

Resource outputs can be found under Attributes Reference in each resource documentation.


Plan (Test) Changes

So far we have been writing code but we have not even tested if they work or not. Through terraform plan we can dry run our scripts against the provider and check if your scripts are okay. Lets to that

Once you run this command, you will get this prompt;

At this point terraform is looking at your variables but it’s not aware where you get the values from. You can input these values one by one, however, this will not be a scalable solution. When your project grows it would be next to impossible to memorize all the values. Instead, we can add the values in a terraform.tfvars file. This will be a go-to file for terraform to look for answers (variables values).

Let’s create the file and add some values. Exit your current command and run the command below.

In this file, add the following values. Modify the values below where necessary especially the email and API key.

Once you have this setup, run terraform plan again. If you configured everything correctly you should see something similar to the output below.

When we run terraform plan, terraform will dry run your configuration against the provider API and verify whether what you have specified is possible. If what we declared is not possible you might get errors, which might range from the wrong resource to unique names such as Heroku apps.

Terraform plan also notifies us of how many resources will be created, updated or destroyed.


Applying Changes

Once we are content with the changes, we can run terraform apply. This command will run the configuration against the provider and provision the resources for us.

When terraform is applying your changes, it will report in real-time the progress of each resource declared. And once complete it also provides a report of resources created, changed, or destroyed.

Once the command has completed executing, visit your heroku dashboard, there should be a pipeline with three apps attached to it.

The pipeline.

The pipeline with 3 apps.

As you have noticed it took very little time from running the command to having your infrastructure up and running. As compared to the manual process. Since this scripts can versioned, we can quickly spin out the same infrastructure anytime and get the same results.


Terraform Outputs

Terraform outputs enables us to get specific values from resources that we might want to use later. This is important when you are working complex infrastructure and you need to share values or you are working with modules. Terraform outputs assume this syntax.

In our setup above we might need to get all the git URLs of our apps. This can be done as below. Place the code below in outputs.tf file.

Since we had applied our changes, we will need to refresh terraform state in order for our outputs to take effect. When you run terraform refresh, terraform will read this file and produce its outputs for you. This outputs can also be displayed at the end of a terraform apply or accessed via terraform output command.

After refreshing terraform state you should see your outputs as below


Destroy Applied changes

In one way or the other, you might want to delete your infrastructure. Terraform provides you will a command terraform destroy. This command will nuke all the resources that were created by terraform apply command

Note: changes made by this command are irreversible and therefore should be done when 100% sure that what you want.


Conclusion

Automation is one of the key principles that DevOps engineers and Developers alike should learn to adopt it in their daily workflow. This approach helps to reduce the time spent on minor tasks and focus on improving and building products. It also introduces flexibility in that you can change and modify your infrastructure much easier as compared going the manual way.

Terraform works well with a lot of platforms and services, making it a perfect tool to use to manage your infrastructure. It can be used across most common services and cloud platforms.

If you would like to have further discussion on how to use terraform feel free to reach out to us on twitter, facebook or via email.

Rack Brains Africa

Our mission is to provide training and awareness about DevOps, its ecosystem and, the importance of designing and building effective products.

Thomas Nyambati

Written by

Full stack developer | DevOps Engineer |Author |Tech Evangelist | Founder @rackbrainz

Rack Brains Africa

Our mission is to provide training and awareness about DevOps, its ecosystem and, the importance of designing and building effective products.

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade