Terraformer — Generate Terraform Files from Existing Infrastructure
What is Terraformer?
Terraformer is a CLI tool , which is written in Go . It is used to generate terraform files from existing infrastructure (Infrastructure to Terraform Files). It’s quite complex that the entire infrastructure begins to be written from scratch. Many industries have existing complex cloud infrastructure setup and they face challenge while integrating their existing infrastructure to Terraform script in a timely and cost-effective manner that Terraformer solves.
Terraformer is a tool which is created by Waze SRE . It is an open-source tool that can be modified and used across all major platforms and it supports supports 16 clouds and more than 15+ providers like Kubernetes, Datadog, PagerDuty, GitHub etc.
To read more about different providers supported by Terraformer , checkout this link below :
Capabilities of Terraformer
1. Generate tf/json + tfstate files from existing infrastructure for all supported objects by resource.
2. Remote State sharing and exporting to specified bucket locations.
3. Filtering
4. Requires only read-only permissions to list service resources, thus ensuring that it does not change anything within the existing infrastructure.
5. Connect between resources with terraform_remote_state (local and bucket).
6. Planning Capabilities — similar to Terraform plan
7. Import by resource name and type.
Steps for Using Terraformer :
1.Install Terraform beforehand(Check terraform version)
2.Installation of Terraformer:
Linux:
export PROVIDER={all,google,aws,kubernetes}
curl -LO https://github.com/GoogleCloudPlatform/terraformer/releases/download/$(curl -s
https://api.github.com/repos/GoogleCloudPlatform/terraformer/releases/latest | grep tag_name | cut -d ‘“‘ -f 4)/terraformer-${PROVIDER}-linux-amd64
chmod +x terraformer-${PROVIDER}-linux-amd64
sudo mv terraformer-${PROVIDER}-linux-amd64 /usr/local/bin/terraformer
MacOs:
export PROVIDER={all,google,aws,kubernetes}
curl -LO https://github.com/GoogleCloudPlatform/terraformer/releases/download/$(curl -s https://api.github.com/repos/GoogleCloudPlatform/terraformer/releases/latest | grep tag_name | cut -d ‘“‘ -f 4)/terraformer-${PROVIDER}-darwin-amd64
chmod +x terraformer-${PROVIDER}-darwin-amd64
sudo mv terraformer-${PROVIDER}-darwin-amd64 /usr/local/bin/terraformer
Using Package Manager:
$brew install terraformer
Terraformer uses Terraform providers under hood and is designed to easily support newly added resources.
$terraformer
$ terraformer plan google -h
In order to access the information from your Google Project, you need to provide authentication credentials by setting up the environment variable GOOGLE_APPLICATION_CREDENTIALS with the file path of the JSON file that contains your service account key.
First Create a service account in GCP IAM (Service Account) with Viewer Role (Basic) :
Create a key in the Service Account and download the JSON file and pass the file path of the JSON file in GOOGLE_APPLICATION_CREDENTIALS variable
$export GOOGLE_APPLICATION_CREDENTIALS = <path of JSON file>
Run terraform init against a init.tf file to install the plugins required for your platform.
terraform {
required_providers {
google = {
source = “hashicorp/google”
}
}
required_version = “>= 0.13”
}
I have created some resources in my GCP Account like VPC, Subnets , GKE cluster, Cloud Storage buckets , Compute Engine, Pub Sub Topic in asia-east-1 and us-east-1 regions.
Created Cloud Storage Buckets with name “devops-demo-bucket-gcs” in us region and “devops-demo-bucket-gcs-asia” in asia region.
Created Compute Engines with name “demo-instance-asia-east-1” in asia-east1-b zone and “demo-instance-us-east” in us-east1-b zone.
Created GKE Cluster with 3 default nodes named as “demo-cluster-terraformer” in us-east1-b zone
Created Cloud Pub/Sub service named as “demo-pubsub-terraformer” which is Global
Using Terraformer import command for generating terraform files from existing infrastructure
$ terraformer import google — resources=networks,pubsub,gke,gcs,project — projects=qwiklabs-gcp-03–7acf96a64118 — regions=us-east1,asia-east1
Where,
terraformer: Invocation of Terraform CLI,
import: Import by resource name and type,
google: Cloud Provider,
— resources parameter: To tell resources from what service you want to import. Example for Cloud Storage we used “gcs”…
Use : --resources="*" if you want to import all services
--projects: The name of the project in GCP where all Google Cloud Resources are organized
--regions: Regions in which you resources are , in GCP Account.
Now, you are able to see the output generated for terraformer import command .
Terraformer by default separates each resource into a file, which is put into a given service directory. The default path for resource files is {output}/{provider}/{service}/{resource}.tf and can vary for each provider.
Terraformer generated many files, let’s see the files and directories structure.
Here, we can see that we get terraform files with tfstate file in every resource’s directory.
To use different resources in GCP go through the below link:
Let’s see some resource’s Terraform files created by Terraformer :
Cloud Storage Bucket :
VPC Networks:
Google Kubernetes Engine(GKE) :
Cloud Pub/Sub Topic:
Cloud Pub/Sub Subscriptions:
Buy me a coffee , if you like my article :)
