Terraform 0.12 Development: A Step-by-Step Guide to Get Running with Providers

6 min readFeb 25, 2019

NOTE: An updated version of these instructions can be found here.

Getting the latest development version of Terraform 0.12 working with semi-separately managed plugins, like the AWS provider, can be a bit tricky.

Kristin Laemmert, a core Terraform engineer, gives a tour of Terraform 0.12

Hashicorp has released Alpha binaries of v0.12 in addition to development snapshots.

If you're itching for something newer, you can try the latest Terraform development version. It’s really easy to build. But, to paraphrase Doc Brown:

No, no, no, Marty. Both you and [Terraform core] turn out fine. It’s your [plugins], Marty. Something’s gotta be done about your [plugins]!

If you want to use AWS or some other provider plugin with your freshly compiled development Terraform 0.12, you might see this error message:

Error: Failed to instantiate provider "aws" to obtain schema: Incompatible API version with plugin. Plugin version: 4, Client versions: [5]

Or, you might see this error message:

Provider "aws" v1.60.0 is not compatible with Terraform v0.12.0.

To use the latest Terraform v0.12.0-dev with a compatible provider, like AWS, you’re going to need some trickery. Thankfully, Mitchell Hashimoto provided some direction, which I embellish here in step-by-step fashion.

1. Install Git and Go

You’re probably covered here but if not, check out these instructions.

2. Uninstall Terraform

Since we’re building a new Terraform, make sure that earlier installs aren’t lurking about.

One of these commands should uninstall Terraform for most people:

$ brew uninstall terraform
Uninstalling /usr/local/Cellar/terraform/0.11.11... (6 files, 102.3MB)
$ rm $GOPATH/bin/terraform
$ rm /usr/local/bin/terraform

Now when you try to check the Terraform version, you should get a not found error:

$ terraform --version
-bash: terraform: command not found

3. Make a home for Terraform

Terraform will need to be in the right place in your GOPATH. Start by making sure some key directories exist and get to the right place:

$ mkdir -p $GOPATH/src/github.com/hashicorp
$ cd $GOPATH/src/github.com/hashicorp

4. Git the latest Terraform

Grab the latest, possibly bleeding-edge, development version of Terraform.

If you do not have a Terraform fork, clone the main repository to your local machine:

$ git clone https://github.com/hashicorp/terraform.git
Cloning into 'terraform'...
remote: Enumerating objects: 9, done.
remote: Counting objects: 100% (9/9), done.
remote: Compressing objects: 100% (7/7), done.
remote: Total 203864 (delta 2), reused 2 (delta 2), pack-reused 203855
Receiving objects: 100% (203864/203864), 153.72 MiB | 20.75 MiB/s, done.
Resolving deltas: 100% (123318/123318), done.
Checking out files: 100% (5912/5912), done.
$ cd terraform

If you have a Terraform fork, clone your fork to your local machine and add an upstream remote to point back to the main repository (replace YakDriver with your Github username):

$ git clone https://github.com/YakDriver/terraform.git
Cloning into 'terraform'...
remote: Enumerating objects: 9, done.
remote: Counting objects: 100% (9/9), done.
remote: Compressing objects: 100% (7/7), done.
remote: Total 203864 (delta 2), reused 2 (delta 2), pack-reused 203855
Receiving objects: 100% (203864/203864), 153.72 MiB | 20.75 MiB/s, done.
Resolving deltas: 100% (123318/123318), done.
Checking out files: 100% (5912/5912), done.
$ cd terraform
$ git remote add upstream https://github.com/hashicorp/terraform.git

Make sure that your fork is at the latest commit:

$ git pull upstream master
From https://github.com/hashicorp/terraform
* branch                master     -> FETCH_HEAD
Already up to date.
$ git push
Everything up-to-date

5. Enable Go modules

Although previously not Go-module-enabled, the latest Terraform is. Enable Go modules with the GO111MODULE environment variable:

$ export GO111MODULE=on

6. Build Terraform?

No, not yet. This is not the best time to build Terraform. When we work on the Terraform AWS provider, it is going to mess with our dependencies and often will introduce incompatibilities (depending on the day). To avoid that, we will wait until after the AWS provider has had it’s fun with the dependencies. Then we’ll build both Terraform and the AWS provider with the same dependencies in place.

7. Grab the latest commit hash

Before moving to the Terraform AWS provider, we are going to need the commit hash for Terraform that we just pulled. We’ll use the hash to make the Terraform AWS provider compatible with our Terraform.

$ git rev-parse --short HEAD
16823f43d

Excellent.

8. Make a home for the AWS provider

Like Terraform core, the AWS provider needs a directory.

$ mkdir -p $GOPATH/src/github.com/terraform-providers
$ cd $GOPATH/src/github.com/terraform-providers

9. Git the latest Terraform AWS provider

Grab the latest, possibly bleeding-edge, development version of the Terraform AWS provider.

If you do not have an AWS provider fork, clone the main repository to your local machine:

$ git clone https://github.com/terraform-providers/terraform-provider-aws.git
Cloning into 'terraform'...
remote: Enumerating objects: 9, done.
remote: Counting objects: 100% (9/9), done.
remote: Compressing objects: 100% (7/7), done.
remote: Total 203864 (delta 2), reused 2 (delta 2), pack-reused 203855
Receiving objects: 100% (203864/203864), 153.72 MiB | 20.75 MiB/s, done.
Resolving deltas: 100% (123318/123318), done.
Checking out files: 100% (5912/5912), done.
$ cd terraform-provider-aws

If you have an AWS provider fork, clone your fork to your local machine and add an upstream remote to point back to the main repository (replace YakDriver with your Github username):

$ git clone https://github.com/YakDriver/terraform-provider-aws.git
Cloning into 'terraform-provider-aws'...
remote: Enumerating objects: 24, done.
remote: Counting objects: 100% (24/24), done.
remote: Compressing objects: 100% (22/22), done.
remote: Total 82582 (delta 11), reused 2 (delta 2), pack-reused 82558
Receiving objects: 100% (82582/82582), 66.49 MiB | 24.17 MiB/s, done.
Resolving deltas: 100% (53542/53542), done.
Checking out files: 100% (4351/4351), done.
$ cd terraform-provider-aws
$ git remote add upstream https://github.com/terraform-providers/terraform-provider-aws.git

Make sure that your fork is at the latest commit:

$ git pull upstream master
From https://github.com/terraform-providers/terraform-provider-aws
* branch                master     -> FETCH_HEAD
Already up to date.
$ git push
Everything up-to-date

10. Go get Terraform dependencies for AWS provider

If we take a look in the go.mod file, we can see which Terraform version the AWS provider was built to work with.

$ cat go.mod | grep "terraform v"
         github.com/hashicorp/terraform v0.11.9-beta1

This will not be compatible with the version of Terraform we’re going to build.

Using Go modules and the commit hash from step 7 above, update the dependency versions:

$ go get -u github.com/hashicorp/terraform@16823f43d
go: finding github.com/hashicorp/terraform 16823f43d
go: finding github.com/gophercloud/gophercloud latest
go: finding github.com/hashicorp/go-plugin latest
go: finding github.com/antchfx/xpath latest
go: finding github.com/agl/ed25519 latest
go: finding github.com/kardianos/osext latest
go: finding github.com/mitchellh/go-linereader latest
...

Checking the go.mod file again, we’ll see Terraform has been updated:

$ cat go.mod | grep "terraform v"
    github.com/hashicorp/terraform v0.12.0-alpha4.0.xx-16823f43deec

11. Build the Terraform AWS provider

Then the provider should build without error:

$ go build

12. Build Terraform

We’re now ready to build Terraform. Using Terraform’s included Makefile, this command will compile Terraform and install it into the Go bin folder.

$ cd $GOPATH/src/github.com/hashicorp/terraform
$ make dev
==> Checking that code complies with gofmt requirements...
GO111MODULE=off go get -u golang.org/x/tools/cmd/stringer
GO111MODULE=off go get -u golang.org/x/tools/cmd/cover
GO111MODULE=off go get -u github.com/golang/mock/mockgen
GOFLAGS=-mod=vendor go generate ./...
2019/02/25 16:56:34 Generated command/internal_plugin_list.go
# go fmt doesn't support -mod=vendor but it still wants to populate 
# the module cache with everything in go.mod even though formatting 
# requires no dependencies, and so we're disabling modules mode for 
# this right now until the "go fmt" behavior is rationalized to 
# either support the -mod= argument or _not_ try to install things.
GO111MODULE=off go fmt command/internal_plugin_list.go > /dev/null
go install -mod=vendor .
$ terraform -v
Terraform v0.12.0-dev

12. Install the AWS provider

We’ll put the newly minted provider in a special place where Terraform will find it. Also, we’ll give it a nonexistent but recognizable version number:

$ mkdir -p ~/.terraform.d/plugins
$ cd $GOPATH/src/github.com/terraform-providers/terraform-provider-aws
$ mv terraform-provider-aws ~/.terraform.d/plugins/terraform-provider-aws_v4.8.15

13. Give it a whirl!

You should now be able to try your HCL against a working Terraform 0.12 development version!

A simple HCL file that uses the AWS provider:

provider "aws" {
  region  = "us-east-1"
}resource "aws_s3_bucket_object" "object1" {
  bucket  = "yak-forsok"
  key     = "arch/three_gossips/turret"
  content = "Delicate"
}

Initialize Terraform:

$ terraform initInitializing provider plugins...The following providers do not have any version constraints in configuration, so the latest version was installed.To prevent automatic upgrades to new major versions that may contain breaking changes, it is recommended to add version = "..." constraints to the corresponding provider blocks in configuration, with the constraint strings suggested below.* provider.aws: version = "~> 4.8"Terraform has been successfully initialized!
...