Terraform Cloud Project Bootcamp with Andrew Brown — 2.1.0 Setup Skeleton for Custom Terraform Provider

This article is part of my Terraform journey with Terraform Bootcamp by Andrew Brown and Andrew Bayko with Chris Williams and Shala.

My wild career adventure into Cloud Computing continues with Andrews, and I highly, highly recommend you to come join us if you are interested. Check out Andrews’ free youtube learning contents. You can also buy some paid courses here to support their good cause.

Agenda

Video here: 2 1 0 Setup Skeleton for Custom Terraform Provider

Issue #44 Goals

• ✅ 1) Write our own Terratowns Provider skeleton in Golang

Workflow

• Conceptual Summary
• Code we write
• ✅ 1) main.go
• ✅ 2) .terraformrc
• ✅ 3) build_provider (bash script)
• ✅ 4) go.mod

Conceptual Summary

In this episode, we will write our own custom provider in Golang. It has to be Go as Terraform Plugins are distributed as Go binaries. Below is a simplified view of the skeleton of our Go plugin. The three dots (…) indicate that there is a lot going on in the final source code.

• In go applications, main() is the entry point for the go program.
• Config holds credential data which is used in Provider().
• The Provider() function instantiates a terraform provider(s) which will be the terratowns.cloud website.
• The Resource() function instantiates a terraform resource(s) which will give shape to our homes in TerraTowns.
• The Resource() function comes with the four CRUD methods which allow us to manipulate the Resource(terratown homes)’s data.

// entry point
func main() { ... }

// Provider handling 
type Config struct { ... }

func Provider() *schema.Provider { ... }
func validateUUID(...) { ... }
func providerConfigure(...) ... { ... }

// Resource handling 
func Resource() ... { ... }

func resourceHouseCreate(...) diag.Diagnostics {...}
func resourceHouseRead(...) diag.Diagnostics {...}
func resourceHouseUpdate(...) diag.Diagnostics {...}
func resourceHouseDelete(...) diag.Diagnostics {...}

Code we write

We will write the following three files.

• main.go
• .terraformrc
• build_provider (bash script)
• go.mod (Andrew manually creates this)

Below are what’s generated automatically by Go.

• go.mod
• go.sum

Let’s dive in.

1. Write main.go

Create a new folder for our custom provider. cd into the folder and create the main.go file. We will write our custom provider in Golang.

mkdir terraform-provider-terratowns
cd terraform-provider-terratowns
go mod init  // Initialise the directory.
code main.go

Below is the go code that will define our custom provider and is the final skeleton at the end of the video 2.1.0. I provide the final version for your convenience. If you are in trouble, feel free to copy and paste (Andrew’s original is here). However, I strongly encourage you to follow along Andrew’s video.

📑 Important notes from Andrew:

• There are schema at the Resource() level and Provider() level.
• Provider() is the terratowns.cloud website and we will need three attributes: endpoint (terratowns.cloud), access token and user_uuid to successfully access the website.
• Resource() is our TerraTown home(s). Every home needs five attributes: name, description, domain_name, town, and content_version.

// packadge main: Declares the package name.
// The main package is special in Go, it's where the execution of the program starts.
package main

// fmt is short format, it contains functions for formatted I/O.
import (
 "fmt"

 "github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema"
 "github.com/hashicorp/terraform-plugin-sdk/v2/plugin"
)

// func main(): Defines the main function, the entry point of the app.
// When you run the program, it starts executing from this function.
func main() {
 plugin.Serve(&plugin.ServeOpts{
  ProviderFunc: Provider,
 })
 // Format.PrintLine
 // Prints to standard output
 fmt.Println("Hello, world")
}

// In golang, a titlecase function will get exported.
func Provider() *schema.Provider {
 var p *schema.Provider
 p = &schema.Provider{
  ResourcesMap:   map[string]*schema.Resource{},
  DataSourcesMap: map[string]*schema.Resource{},
  Schema: map[string]*schema.Schema{
   "endpoint": {
    Type:        schema.TypeString,
    Required:    true,
    Description: "The endpoint for the external service",
   },
   "token": {
    Type:        schema.TypeString,
    Sensitive:   true, // make the token as sensitive to hide in the logs.
    Required:    true,
    Description: "Bearer token for authorisation",
   },
   "user_uuid": {
    Type:        schema.TypeString,
    Required:    true,
    Description: "UUID for configuration",
    // ValidateFunc: validateUUID,
   },
  },
 }
 // p.ConfigureContextFunc = providerConfigure(p)
 return p
}

// func validateUUID(v interface{}, k string) (ws []string, errors []error) {
//  log.Print('validateUUID:start')
//  value := v.(string)
//  if _, err = uuid.Parse(value); err != nil {
//   errors = append(error, fmt.Errorf("invalid UUID format"))
//  }
//  log.Print('validateUUID:end')
// }

2. .terraformrc

By convention, we need .terraformrc because terraform’s default way to install provider plugins is from a provider registry. As it won’t be able to find our custom provider plugin from the Hashicorps’ registry (registry.terraform.io), we have to overwrite the path to the Registry and provide our custom local path.

code .terraformrc

# .terraformrc
provider_installation {
    filesystem_mirror {
        path = "/home/gitpod/.terraform.d/plugins"
        include ["local.providers/*/*"]
    }
    direct {
        exclude = ["local.providers/*/*"]
    }
}

• Andrew referred to this article for this section. Have a look for more information 👀.

3. build_provider

Go is a compiled language, meaning it needs to be translated into machine code so the processor can execute it. In case it doesn’t make sense, just know that we need to run the command “go build” to produce a compiled binary file (not human-readable).

The build_provider file below is the bash script that automates the process of compiling the main.go file we wrote for our custom provider. The script’s workflow is:

• 1) Define paths
• 2) Copy .terraformrc
• 3) Delete existing binary file: every time we make changes to the source code (main.go), we have to manually rebuild it by running the go build command. So we have to remove the existing binary first.
• 4) Build then copy to specified locations

cd $PROJECT_ROOT/bin
code build_provider

# build_provider
#! /usr/bin/bash

# 1) Define path 
PLUGIN_DIR="~/.terraform.d/local.providers/local/terratowns/1.0.0"
PLUGIN_NAME="terraform-provider-terratowns_v1.0.0"

# 2) Copy .terraformrc
cd $PROJECT_ROOT/terraform-provider-terratowns
cp $PROJECT_ROOT/terraformrc /home/gitpod/.terraformrc

# 3) Delete existing binary file
rm -rf ~/.terraform.d/plugins
rm -rf $PROJECT_ROOT/.terraform
rm -rf $PROJECT_ROOT/.terraform.lock.hcl

# 4) Build then copy to specified locations
go build -o $PLUGIN_NAME
mkdir -p $PLUGIN_DIR/x86_64/
mkdir -p $PLUGIN_DIR/linux_amd64/
cp $PLUGIN_NAME $PLUGIN_DIR/x86_64 
cp $PLUGIN_NAME $PLUGIN_DIR/linux_amd64

📑 Andrew’s notes: x86_64 and amd64

When we build binaries, we have to build them based on the specific chipset of our computer. The chipset type (example: x86_64, linux_amd64) can vary depeding on your computer’s OS and model (Windows, Mac, Linux, etc). Hence Andrew is just putting both x86_64 andlinux_amd64 to make sure it works as it’s either of the two for our gitpod workspace.

4. go.mod

go.mod is automatically genrated when we run go mod init. Andrew found the generated file useless (😆) so he decides to write it on our own.

• replace function replaces PROJECT_PATH to whatever path you specify.
• replace [PROJECT_PATH] => [YOUR_LOCAL_PATH]

// go.mod
module github.com/ExamProCo/terraform-provider-terratowns

go 1.20

// This is specific to our local machine.
replace github.com/ExamProCo/terraform-provider-terratowns => /workspace/terraform-beginner-bootcamp-2023/terraform-provider-terratowns

Andrew tests compiling the custom provider by running the command:

go build -o terraform-provider-terratowns_v1.0.0

The binary has been generated. Our code works!

In case you are missing a plugin, go will tell you it’s missing and give you the command to install it. The code looks like this:

go get github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema

Notes

• In the context of Terraform, a “provider” is a way to interact with APIs and resources.
• There’s a schema at the provider level (terratowns.cloud) and schema at the resource level (terratown homes).
• Schemas have the fields that we have to fill to define the provider and resource.
• go mod init initialises the directory for our terraform provider in Go (this is how you initialise any go project).
• The primary purpose of using the import fmt is to use formatted go IO functions.
• go build generates a binary file (.bin).

Resources

Bootcamp

• Video: 2 1 0 Setup Skeleton for Custom Terraform Provider
• My feature branch: 2 1 0: 47-terratowns-provider
• My complete Learning Journal

Terraform

• Call APIs with Custom SDK Providers
• Writing Custom Terraform Providers

Brilliant

• Terraform Local Providers and Registry Mirror Configuration