Creating A Terraform Provider — Part 2

Nathan Mclean
Feb 20 · 6 min read

In part 1 of Creating a Terraform Provider, we walked through how to implement the functionality of a provider; we have a Provider that can create, update and delete ‘items’ by interacting with a simple API. In part 2 we’ll look at how to test the provider as well as how to run it.

Testing

Hashicorp provides helper functions to aid you in testing your provider. One such function is InternalValidate, which runs some checks on your provider. We can add this test to a file called provider_test.go.

To this file, we can add an init function which will initialise our provider for use in testing our Item resource.

Lastly, we can add a function that will check that we have all the configuration, in the form of environment variables, that we need to connect to a running instance of a server.

As we have the code for the server we could run it in memory and use setup and teardown functions in the tests, but you won’t always have this option, so we’ll test against a server that is started outside of the tests.

Now we can start testing our Item resource. Hashicorp has provided more helper functions on the resource Type that make testing resources much easier. We can set up a resource.TestCase and provide some setup functions and test cases and let Terraform take care of the rest:

  • Creating the plan
  • Applying the plan
  • Another plan again — to check that there are no changes left to apply (which will fail the test)
  • Checking the resource exists
  • Checking that the Terraform state is correct
  • Tidy up the resource(s) we created.

Let’s look at a test for our Item resource. These tests will live in resource_item_test.go.

You can see that we set up a resource.Test and provide it with the following:

  • PreCheck — This is the testAccPreCheck function that we created in provider_test.go which will check our environment variables are set.
  • Providers — This is a map of providers, containing our example provider that we set up in the init function of provider_test.go
  • CheckDestroy — This is a function that will check that the item is successfully tidied up at the end of the test and also acts as a test for the resourceDeleteItem function in resource_item.go
  • Steps — A number of test steps to run, in this case, we only have one, we’ll see examples with multiple steps later

In each Step of a test we there are a couple of things to provide:

  • Config — This is a string containing Terraform (HCL) code. I’m using a function that returns this
  • Check — A function (of type resource.ComposeTestCheckFunc) which takes a number of TestCheckFunc — some like TestCheckResourceAttr are provided as helpers but we can also write our own such as testAccCheckExampleItemExists.

Let’s have a look at the various components of the test in more detail.

The testAccCheckItemDestroy is run at the end of our test after all of the Steps have been run and the resources have been destroyed. The function retrieves our client from the provider (testAccProvider) we set up using the init function in provider_test.go. Then for each Resource in the state s it checks if the resource exists. When we make an apiClient.GetItem call we expect a not found error to be returned, otherwise, we return an error which fails the test.

The testAccCheckItemBasic function returns a string containing some Terraform code. This is what the Test will run. Note that by default the tests will be run in parallel, so it’s important to give these test resource unique names so that they don’t collide on the server and give you unexpected errors.

testAccCheckExampleItemExists checks that the Item we asked the test to create was actually created, this is run before the testAccCheckItemDestroy function gets run. It makes a GetItem request to the server and expects not to receive an error.

We also use a built-in check, TestCheckResourceAttr, which takes the resource we want to test, the attribute name and the value we expect that attribute to be and verifies that the value is correct in state.

Where you have a List or Set you can check the length by using the # value. For a List, you can check the value of each element by providing the index, eg tags.0.

For Sets there is a hash of the value that’s stored used as the id to look up, eg tags.1931743815. To find out the hash I’ve tried using the schema.HashString function, which seems to be what is used to generate the hash, but we get a different result back. I’ve ended up just printing out the resource (fmt.Println(rs.Primary)) in the testAccCheckExampleItemExists function and then using that.

Testing Resource Updates

We can also test updating a resource. This can be done using multiple test steps. The first step is exactly as we have done before with the TestAccItem_Basic.

The second step has a different Config value which updates the resource created during the first step. We use testAccCheckItemUpdatePost which updated the description. We can then pass in a number of checks to verify that the resource exists and all of the values are as we expect, including the updated description.

Another test is one that validates that the validateName function is correctly applied to the name attribute. In this test, we apply some Config, which is a resource with spaces in its name and tells the TestStep to expect to fail by providing a regex that should match the error that Terraform returns to the ExpectError field. We can provide the regex whiteSpaceRegex which, is defined in a variable.

The final test I’ve created for this provider is one that tests the import function. This test sits is a separate file; import_resource_test.go

This test starts the same way as the others; we apply some configuration to create a resource. In a second Step, we then tell Terraform to import that resource and verify the state.

Running the Tests

Note: I’m using Go modules for this project so you’ll need to be using a Go version ≥ 1.11 and have the environment variable export GO111MODULE=on set to be able to run the API server, tests and build the provider. Alternatively, you can use a tool like dep to vendor the dependencies.

To run the tests you first need to start the API server — go run api/main.go

Then, in a separate terminal session, we can run our tests:

TF_ACC=true SERVICE_ADDRESS=http://localhost SERVICE_PORT=3001 SERVICE_TOKEN=superSecret go test -v provider/*

TF_ACC tells Terraform that we’re running acceptance tests. We also set the environment variables that our provider requires to set up a client. Then we run all of the tests in the provider package. Terraform requires that the -v flag is applied when TF_ACC is true.

TF_ACC is used to differentiate Acceptance tests from Unit tests as you won’t always want to test against your external dependencies.

Running The Provider

Usually when you run a provider Terraform will go off and download it for you, but this is only possible with officially supported providers. To run your own provider it must be stored in the ~/.terraform.d/plugins/[distro] directory on Linux/OSX or in terraform.d\plugins in your user's "Application Data" directory on Windows. The provider needs to be named in a specific way; terraform-provider-[providerName]_[version] where version is a semantic version in the form vX.X.X.

Running make in the root of the sample code will build the provider, which you can then move the correct location. The version comes from the version file. This plugin file won’t work for people on different platforms though, you’ll need to compile it separately for each platform people may use it from.

Once you have your plugin built and in the correct location you can create a terraform file similar to this one:

Next, we can start the server, to do this run make startapi or go run api/main.go which will start a server on localhost:3001.

Now we can run terraform init, terraform plan and terraform apply to see our provider in action.

Some Final Advice

Two final pieces of advice to leave you with:

  1. Reading through the code for the of the official providers can be a really useful way to learn how to write your own. All of the official providers live in the terraform-providers organisation on GitHub
  2. The Terraform Go Docs are well written, so I recommend spending some time reading through the docs for the helpers that Hashicorp provides, for example, the resource helper. For instance, the TestStep explains how to use each of the fields, including some I haven’t mentioned, such as Taint, which will let you test that your provider works as expected with a tainted resource.

spaceapetech

Space Ape Technology

50

50 claps
Nathan Mclean

Written by

DevOps Engineer for Space Ape Games based in London

spaceapetech

Space Ape Technology