Open in app

Sign in

Write

Sign in

How I used Google Cloud Storage to host my own Terraform providers registry

Jérôme NAHELOU
CodeShake
Published in
6 min readOct 21, 2020

If you don’t want to expose publicly your provider, you should read this.

Access logs from my GCS registry

I enjoy every day my Terraform deployments.

Working with companies having hundreds of deployments over multiple cloud providers and managed services occurred challenges to share code, provider, improve skills or to create an internal community.

All those things push me to share about Terraform. Also this is why I wanted to write this paper.

When you start to use or study Terraform, you’re facing these questions

Is it restricted to cloud resources ?
How to manage resources into my own software ?

These questions can be addressed by an already existing provider (like the generic REST provider[1]), but mostly developing a new provider brings you more comfort, validations or tests like attributes validation, environment variables configuration or default values and challenges are how to share this custom provider.

This article assumes you understand the concept of third-party providers already

Why use a registry ?

In January, @mitchellh tweeted about the new public registry available for all providers (official and community).

https://twitter.com/mitchellh/status/1217495112369917958

Thanks to that, it helps developers to consume third-party providers easily with less than 5 lines of configuration in the manifest versus painful manual download and configuration previously.

But how to host providers developed for a company which uses private custom apis ?

Should we share them publicly ? How to do if you are not a Terraform Cloud user ?

(unpopular opinion) I don’t think releasing non-usable, testing or wip providers will help the community. There are already a lot of community providers which are working well.

Fortunately Terraform provides the documentation to create your own registry[3].

Let’s dig in !

Step 1 : Build

To build our provider, Hashicorp provides 3 different ways[3].
Personally I prefer the 2nd method using goreleaser locally.

In our case, goreleaser will help us to compile cross platform, zip binaries and to sign them with my GPG key in one command.

As described in the documentation[4], to create a Terraform provider usable in the registry, all providers must be signed and goreleaser will do it for me.

You have to start by creating your own key (you can follow github documentation [5]) and then set the fingerprint in environment variable GPG_FINGERPRINT.

If you already have your own key, you can skip the creation step.

$ gpg --default-new-key-algo rsa4096 --gen-key
$ export GPG_FINGERPRINT="C442EB02E6A71004A961576D937C460284513EBB"

Now we can start the configuration of goreleaser itself. It’s a simple config file put directly in your provider project.

Get a template of .goreleaser.yml (https://github.com/hashicorp/terraform-provider-scaffolding/blob/master/.goreleaser.yml)

Set build targets, architectures and uncomment the release block if you want to update the remote repository.

release:
  draft: true

Your repository must be cleaned before compilation. Add, commit or ignore not staged or untracked files. Then run goreleaser !

$ git tag -f v0.0.1
$ goreleaser release --rm-dist

Personally, I set a dummy github token to avoid misconfigurations during local tests.

Once the compilation is successful, verify integrity.

$ gpg --verify dist/terraform-provider-hello_0.0.1_SHA256SUMS.sig dist/terraform-provider-hello_0.0.1_SHA256SUMS
gpg: Signature made ven. 16 oct. 2020 11:43:08 CEST
gpg:                 using RSA key C442EB02E6A71004A961576D937C460284513EBB
gpg: Good signature from "gpg owner  <gpg-owner@mailbox.com>" [ultimate]

It’s time to create the registry !

A Terraform registry can be a website in HTTPS. HTTP is not supported yet.
To host static files, I like to use storage object platforms with static website hosting feature enabled.
It provides a simple way to expose files at low cost with a lot of cool features (versioning, lifecycle, ACLs, audit & logging, …).

I used Google Cloud Storage[6], but similar deployments can be done on AWS S3 for example.

Here is an example of tree required by the registry protocol[4]. We will see later that the binaries can be stored in an other repository.

I simplified the configuration to provide only the linux_amd64 provider. Other target platforms must be configured as well.

Let’s take a look on configuration files

jnahelou/hello/0.0.1/download/linux/amd64

This file (also refered as PackageMeta object in the source code) contains the properties to download and verify your provider. To fill it :

  1. Configure URLs with your own registry
  2. Set the sha256sum for the zip file (you can find it in terraform-provider-hello_0.0.1_SHA256SUMS file)
  3. Add your GPG key ID
  4. Set the public key (hint: use sed to replace newlines)
$ # Get the public key
$ gpg — output public.pgp — armor — export C442EB02E6A71004A961576D937C460284513EBB$ # Dump it in a single line
$ cat public.pgp | sed -E ':a;N;$!ba;s/\r{0,1}\n/\\n/g'

Continue the configuration with the `jnahelou/hello/versions`and .well-known/terraform.json files as described in the documentation.

Once all files are set, put the content on the web server. On GCP, I used gsutil to synchronize the content. Btw I lost 2 hours to understand why randomly gpg checks failed…After some stackoverflow searches I found that despite Cloud CDN were disabled, GCP doesn’t guaranted direct file access.. So I choosed to disable cache during my tests.

$ gsutil -h Cache-Control:"Cache-Control:private, max-age=0, no-transform" rsync -R . gs://registry.jnahelou.net/

Then enable HTTPS, configure DNS, and verify the content.

Authn and Authz

Now that registry works publicly, I used the default Cloud Storage Oauth2 for authentication.
Note that tokens can be provided in the terraformrc configuration file

$ cat ~/.terraformrccredentials "registry1.jnahelou.net" {
  token = "ya29.token-content"
}

Because this token has a 1h lifetime, for users who don’t want to mirror registry content, It could be great to look forward to Credentials Helpers[7]

Wait… Using TF_LOG=TRACE during init step, I Can see the download of providers and signs failed due to 403 errors…

In my version (0.13.4), I found that the checksum files and the providers archives are downloaded without authentication. I think Terraform developers did that to allow the download of files from an other URL (with an other authentication method, that’s why I suppose the token is not propagated).

https://github.com/hashicorp/terraform/blob/59b116f7bfa857def1c7b221f6cf1f6317a29708/internal/getproviders/registry_client.go#L438

But if I replace this code with an authenticated request, it works as expected

Here is my patch to allow the terraform providers mirror command to download all files using the same authentication (not working for terraform init, only patched for terraform providers mirror command)

https://gist.github.com/jnahelou/02e0f727ae298a06933f9f85481bb398

Keep in mind that, using official Terraform release, you have to deal with authentication in hostname(login:password) or URL query parameters (like signed URL) in the package configuration file.

Usage of automatic detection from go-getter could be awesome to allow access for sources hosted on S3, GCS, …

Hope the Terraform team will improve it soon !

Let’s play, Terraform 0.13

Since 0.13, you can configure which registry to use to find and download providers[2].

Automatic Installation of Third-Party Providers with Terraform 0.13

This blog will discuss the introduction of a new Terraform provider source attribute that allows you to declare the…

www.hashicorp.com

Here is how I used my terraform-provider-hello to create a new file on my laptop. At Sfeir, we use this provider during courses to learn how to create a third party provider.

As you can see, the source attribute is composed by my registry hostname followed by my namespace/provider name.

Now I can mirror my HTTP registry on my local disk if needed (for offline purpose for example) and use it as a official provider :)

$ terraform providers mirror offline$ terraform init
/* … */* registry1.jnahelou.net/jnahelou/hello: version = "~> 0.0.1"$ terraform applyhello_file.foo: Creating…
hello_file.foo: Creation complete after 0s [id=/tmp/foo]Apply complete! Resources: 1 added, 0 changed, 0 destroyed.

It’s not new, Hashicorp teams provide very powerful products, from Vagrant to Boundary, for those who want to use them correctly.
The success of these products comes from the ability to extend them. With third party providers registry, Terraform improve the experience and encourage developers to make their own providers.

Object storage platform offers a good trade off between low prices and high resilience to host your providers.

What a great job from Terraform teams since 0.12 release !

Congratulations !

Links

Helpful resources in the journey

Terraform
Hashicorp
Google Cloud Platform
DevOps
Coding

--

--

Jérôme NAHELOU
CodeShake

Written by Jérôme NAHELOU

54 Followers
Writer for

Cloud rider at SFEIR the day, Akita Inu lover #MyAkitaInuIsNotAWolf

Teams