Terraform ! Terraform !! Terraform !!!

A Complete Guide for who wants to take HashiCorp Infrastructure Automation Certification.

Vikram Shinde
Nov 7, 2020 · 19 min read
Image for post
Image for post

The Terraform Associate certification is for Cloud Engineers specializing in operations, IT, or developers who know the basic concepts and skills associated with open source HashiCorp Terraform. Candidates will be best prepared for this exam if they have professional experience using Terraform in production, but performing the exam objectives in a personal demo environment may also be sufficient.

Training Guide and Recommendations

Following are my notes while studying for the Terraform Certification exam.

What is IaC?

Why IaC?

Businesses are making a transition where traditionally-managed infrastructure can no longer meet the demands of today’s businesses. IT organizations are quickly adopting the public cloud, which is predominantly API-driven.

To meet customer demands and save costs, application teams are architecting their applications to support a much higher level of elasticity, supporting technology like containers and public cloud resources. These resources may only live for a matter of hours; therefore the traditional method of raising a ticket to request resources is no longer a viable option.

Benefits of IaC

Speed

Consistency

Cost

Minimum Risk

Everything Codified

Version Controlled, Integrated

IaC allows you to track and give insight on what, who, when, and why anything changed in the process of deployment. This has more transparency which we lack in traditional infrastructure management.

Now, we know what is Infrastructure as Code means, now let’s deep dive into Terraform...

Terraform

Provider

In order to make a provider available on Terraform, we need to make a terraform init, these commands download any plugins we need for our providers. If for example, we need to copy the plugin directory manually, we can do it, moving the files to .terraform.d/plugins

provider "aws" {
region = "us-east-1"
}

If the plugin is already installed, terraform init will not download again unless to upgrade the version, run terraform init -upgrade.

Multiple Providers

#default configuration
provider "aws" {
region = "us-east-1"
}

# reference this as `aws.west`.
provider "aws" {
alias = "west"
region = "us-west-2"
}

Versioning

terraform {
required_version = ">= 0.12"
}

The value for required_version is a string containing a comma-separated list of constraints. Each constraint is an operator followed by a version number, such as > 0.12.0. The following constraint operators are allowed:

  • = (or no operator): exact version equality
  • !=: version not equal
  • >, >=, <, <=: version comparison, where “greater than” is a larger version number
  • ~>: pessimistic constraint operator, constraining both the oldest and newest version allowed. For example, ~> 0.9 is equivalent to >= 0.9, < 1.0, and ~> 0.8.4, is equivalent to >= 0.8.4, < 0.9

We can also specify a provider version requirement

provider "aws" {
region = "us-east-1"
version = ">= 2.9.0"
}

Terraform Workflow

Image for post
Image for post
Basic flow

Terraform Init

Terraform plan

Terraform Apply

Terraform Refresh

Terraform Destroy

You can remove the resource block from the configuration and run terraform apply this way you can destroy the infrastructure.

Terraform Validate

Provisioners

file Provisioner

resource "aws_instance" "web" {
# ...

# Copies the myapp.conf file to /etc/myapp.conf
provisioner "file" {
source = "conf/myapp.conf"
destination = "/etc/myapp.conf"
}

local-exec Provisioner

resource "aws_instance" "web" {
# ...
provisioner "local-exec" {
command = "echo The server's IP address is ${self.private_ip}"
}
}

remote-exec Provisioner

resource "aws_instance" "web" {
# ...

provisioner "remote-exec" {
inline = [
"puppet apply",
"consul join ${aws_instance.web.private_ip}",
]
}
}

Creation-time Provisioners

Destroy-time Provisioners

Other Sub-commands

Terraform Format

Terraform Taint

Once a resource is marked as tainted, the next plan will show that the resource will be destroyed and recreated and the next application will implement this change.

For multiple sub-modules, the following syntax-based example can be used

module.foo.module.bar.aws_instance.baz

Terraform Untaint

Terraform Import

This allows you to take resources that you’ve created by some other means and bring them under Terraform management.

The current implementation of Terraform import can only import resources into the state. It does not generate a configuration.

Because of this, prior to running terraform import, it is necessary to write a resource configuration block manually for the resource, to which the imported object will be mapped.

terraform import aws_instance.myec2 instance-id

Terraform Show

terraform show -json will show a JSON representation of the plan, configuration, and current state.

Terraform plan -destroy

Variables

Variable Types

The type argument in a variable block allows you to restrict the type of value that will be accepted as the value of a variable.

variable "vpcname" {
type = string
default = "myvpc"
}

List: List is the same as array. We can store multiple values Remember the first value is the 0 position. For example, to access the 0 position is var.mylist[0]

variable "mylist" {
type = list(string)
default = ["Value1", "Value2"]
}

Map: Map is a Key-Value pair. Key is needed to access the value.

variable "ami_ids" {
type = map
default = {
"mumbai" = "image-abc"
"germany" = "image-def"
"paris" = "image-xyz"
}
}

use var.ami_ids["paris"] to fetch the corresponding value.

Structural Data Types

List contains multiple values of the same type while objects can contain multiple values of different types.

Input Variables

variable "image_id" {
type = string
}

Input variables are created by a variable block, but you reference them as attributes on an object named var.

resource "aws_instance" "example" {
instance_type = "t2.micro"
ami = var.image_id
}

Because the input variables of a module are part of its user interface, you can briefly describe the purpose of each variable using the optional description argument:

variable "image_id" {
type = string
description = "The id of the machine image (AMI) to use for the server."
}

The description should concisely explain the purpose of the variable and what kind of value is expected.

Assigning Values to Input Variables

  1. Manually set a variable when we run Terraform plan
    If values are set, then it will ask at runtime.
Image for post
Image for post

2. CLI Variables
To specify individual variables on the command line, use the -var option when running the terraform plan and terraform apply commands:

terraform apply -var="image_id=ami-abc123"
terraform apply -var='image_id_list=["ami-abc123","ami-def456"]'
terraform apply -var='image_id_map={"us-east-1":"ami-abc123","us-east-2":"ami-def456"}'

3. TFVARS files

To set lots of variables, it is more convenient to specify their values in a variable definitions file (with a filename ending in either .tfvars or .tfvars.json) and then specify that file on the command line with -var-file:

terraform apply -var-file="testing.tfvars"

4. Auto tfvars files
Terraform also automatically loads a number of variable definitions files if they are present:

  • Files named exactly terraform.tfvars or terraform.tfvars.json.
  • Any files with names ending in .auto.tfvars or .auto.tfvars.json.

5. Environment Variables
As a fallback for the other ways of defining variables, Terraform searches the environment of its own process for environment variables named TF_VAR_ followed by the name of a declared variable.

$ export TF_VAR_image_id=ami-abc123
$ terraform plan

Variable Definition Precedence

  • Environment variables
  • The terraform.tfvars file, if present.
  • The terraform.tfvars.json file, if present.
  • Any *.auto.tfvars or *.auto.tfvars.json files, processed in lexical order of their filenames.
  • Any -var and -var-file options on the command line, in the order they are provided.

If the same variable is assigned multiple values, Terraform uses the last value it finds.

Output Values

resource "aws_instance" "myec2" {
ami = var.image_id
instance_type = "t2.micro"
}
output "instance_id" {
value = aws_instance.myec2.id
}

If we run a apply we can see the next message:

Image for post
Image for post

Local Values

The expression of a local value can refer to other locals, but as usual reference cycles are not allowed. That is, a local cannot refer to itself or to a variable that refers (directly or indirectly) back to it.

It’s recommended to group together logically-related local values into a single block, particularly if they depend on each other.

locals {
# Ids for multiple sets of EC2 instances, merged together
instance_ids = concat(aws_instance.blue.*.id, aws_instance.green.*.id)
}

Data Source

data "aws_ami" "example" {
most_recent = true

owners = ["self"]
tags = {
Name = "app-server"
Tested = "true"
}
}

Reads from a specific data source (aws_ami) and exports results under “app_ami”

Dependencies

resource "aws_instance" "example" {
ami = "ami-a1b2c3d4"
instance_type = "t2.micro"
depends_on = [aws_iam_role_policy.example]
}

Workspace

The workspace feature of Terraform allows users to switch between multiple instances of a single configuration with a unique state file.

For local states, Terraform stores the workspace states in a directory called terraform.tfstate.d.

Workspace commands

  1. The terraform workspace new command is used to create a new workspace and switched to a new workspace.
  2. The terraform workspace list command is used to list all existing workspaces.
  3. The terraform workspace select command is used to choose a different workspace to use for further operations.
  4. The terraform workspace delete command is used to delete an existing workspace.
  5. The terraform workspace show command is used to output the current workspace.

Note: Terraform Cloud and Terraform CLI both have features called “workspaces,” but they’re slightly different.

States

Terraform uses state to keep track of the infrastructure it manages. To use Terraform effectively, you have to keep your state accurate and secure.

State is a necessary requirement for Terraform to function. It is often asked if it is possible for Terraform to work without state, or for Terraform to not use state and just inspect cloud resources on every run.

Terraform requires some sort of database to map Terraform config to the real world. Alongside the mappings between resources and remote objects, Terraform must also track metadata such as resource dependencies. Terraform stores a cache of the attribute values for all resources in the state. This is done to improve performance.

For small infrastructures, Terraform can query your providers and sync the latest attributes from all your resources. This is the default behavior of Terraform: for every plan and application, Terraform will sync all resources in your state.

For larger infrastructures, querying every resource is too slow. Larger users of Terraform make heavy use of the -refresh=false flag as well as the -target flag in order to work around this. In these scenarios, the cached state is treated as the record of truth.

State Management

Terraform has a force-unlock command to manually unlock the state if unlocking failed.

Sensitive Data
Terraform state can contain sensitive data, e.g. database password, etc.
When using a remote state, the state is only ever held in memory when used by Terraform.
The S3 backend supports encryption at rest when the encrypt option is enabled. IAM policies and logging can be used to identify any invalid access. Requests for the state go over a TLS connection.

Note: Setting an output value in the root module as sensitive prevents Terraform from showing its value in the list of outputs at the end of terraform apply. However, output values are still recorded in the state and so will be visible to anyone who is able to access the state data.

output "db_password" {
value = aws_db_instance.db.password
description = "The password for logging in to the database."
sensitive = true
}

Backend Management

Terraform must initialize any configured backend before use.

Local
By default, Terraform uses the “local” backend. After running first terraform apply the terraform.tfstate file created in the same directory of main.tf

Image for post
Image for post

terraform.tfstate file contains JSON data.
The local backend stores state on the local filesystem, locks the state using system APIs, and performs operations locally.

terraform {
backend "local" {
path = "relative/path/to/terraform.tfstate"
}
}

Remote
When working with Terraform in a team, the use of a local file makes Terraform usage complicated because each user must make sure they always have the latest state data before running Terraform and make sure that nobody else runs Terraform at the same time.

With a remote state, Terraform writes the state data to a remote data store, which can then be shared between all members of a team.

terraform {
backend "remote" {}
}

This is called partial configuration

When configuring a remote backend in Terraform, it might be a good idea to purposely omit some of the required arguments to ensure secrets and other relevant data are not inadvertently shared with others.

terraform init -backend-config=backend.hcl

Standard Backend Types
AWS S3 bucket.
AWS S3 is typically the best bet as a remote backend for the following reason
* It’s a managed service, so no need to manage infrastructure.
* It supports encryption at rest.
* It support locking via DynamoDB
* It supports versioning, so you can roll back to an older version.

terraform {
backend "s3" {
bucket = "mybucket"
key = "path/to/my/key"
region = "us-east-1"
dynamodb_table = "terraform-locks"
encrypt = true
}
}

Terraform will automatically detect that you already have a state file locally and prompt you to copy it to the new S3 backend. If you type in “yes,” you should see:

Successfully configured the backend "s3"! Terraform will automatically use this backend unless the backend configuration changes.

After running this command, your Terraform state will be stored in the S3 bucket.

Note: GitHub is not supported as backend type

Terraform State commands

terraform state mv : Move items within terraform state. This will be used to resource renaming without destroy, apply command

terraform state pull : Manually download and output the state from the state file.

terraform state push : Manually upload a local state file to the remote state

terraform state rm : Remove items from the state. Items removed from the state are not physically destroyed. This item no longer managed by Terraform.

terraform state show Show attributes of a single resource in the state.

Modules

Calling Child Modules

Input variables to accept values from the calling module.
Output values to return results to the calling module, which it can then use to populate arguments elsewhere.
Resources to define one or more infrastructure objects that the module will manage.

variable "image_id" {
type = string
}
resource "aws_instance" "myec2" {
ami = var.image_id
instance_type = "t2.micro"
}

output "instance_ip_addr" {
value = aws_instance.myec2.private_ip
}

Call to the module example:

module "dbserver" {
source = "./db"
image_id = "ami-0528a5175983e7f28"
}

Module outputs are very similar to module inputs, an example in a module output:

output "privateip" {
value = aws_instance.myec2.private_ip
}

It is recommended to explicitly constraining the acceptable version numbers for each external module to avoid unexpected or unwanted changes.

Version constraints are supported only for modules installed from a module registry, such as the Terraform Registry or Terraform Cloud’s private module registry.

Debugging in Terraform

You can set TF_LOG to one of the log levels TRACE, DEBUG, INFO, WARN or ERROR to change the verbosity of the logs.

export TF_LOG=TRACE

To persist logged output, you can set TF_LOG_PATH

TF_LOG_PATH=./terraform.log

Terraform Functions

max(5, 12, 9)
12

The Terraform language does not support user-defined functions, and so only the functions built into the language are available for use

Some other example built-in functions

element retrieves a single element from a list.

element(["a", "b", "c"], 1)
b

lookup retrieves the value of a single element from a map, given its key

lookup({a="ay", b="bee"}, "c", "what?")
what?

Count and Count Index

resource "aws_instance" "myec2" {
ami = var.image_id
instance_type = "t2.micro"
count = 3
}

output "instance_ip_addr" {
value = aws_instance.myec2[*].private_ip
}
Image for post
Image for post

Terraform Cloud

The remote backend stores Terraform state and may be used to run operations in Terraform Cloud. When using full remote operations, operations like terraform plan or terraform apply can be executed in Terraform Cloud’s run environment, with log output streaming to the local terminal.

Sentinel is an embedded policy-as-code framework integrated with the HashiCorp Enterprise products.
Sentinel is a proactive service.

Note: Terraform Cloud always encrypts the state at rest and protects it with TLS in transit.

Terraform Enterprise

Some of these include:

  • Single Sign-On
  • Auditing
  • Private Data Center Networking
  • Clustering

Team & Governance features are not available for Terraform Cloud Free (Paid)

Miscellaneous

  1. The existence of a provider plugin found locally in the working directory does not itself create a provider dependency. The plugin can exist without any reference to it in Terraform configuration.
  2. The overuse of dynamic blocks can make configuration hard to read and maintain.
  3. By default, provisioners that fail will also cause the terraform to apply itself to fail. The on_failure=continue setting can be used to change this.
  4. Terraform Enterprise requires a PostgresSQL for a clustered deployment.
  5. Terraform can limit the number of concurrent operations as Terraform walks the graph using the -parallelism=n argument. The default value for this setting is 10. This setting might be helpful if you’re running into API rate limits.
  6. terraform.tfstate and *.tfvars contains sensitive data, hence should be configured in .gitingore file.
  7. Arbitrary Git repositories can be used by prefixing the address with the special git:: prefix.
  8. By default, Terraform will clone and use the default branch (referenced by HEAD) in the selected repository. You can override it with ?ref=version
  9. The terraform console command provides an interactive console for evaluating expressions.

Conclusion

I wish you all the very best on the exam. If you have any questions about the exam, please reach to me.

Happy Terraforming!

Thank you very much for reading.

References

Image for post
Image for post

👋 Join FAUN today and receive similar stories each week in your inbox! Get your weekly dose of the must-read tech stories, news, and tutorials.

Follow us on Twitter 🐦 and Facebook 👥 and Instagram 📷 and join our Facebook and Linkedin Groups 💬

Image for post
Image for post

If this post was helpful, please click the clap 👏 button below a few times to show your support for the author! ⬇

FAUN

The Must-Read Publication for Creative Developers & DevOps Enthusiasts

Sign up for FAUN

By FAUN

Medium’s largest and most followed independent DevOps publication. Join thousands of aspiring developers and DevOps enthusiasts Take a look

By signing up, you will create a Medium account if you don’t already have one. Review our Privacy Policy for more information about our privacy practices.

Check your inbox
Medium sent you an email at to complete your subscription.

Vikram Shinde

Written by

Cloud & DevOps Enthusiast. My old blogs can be found at below links http://vikshinde.blogspot.com/

FAUN

FAUN

The Must-Read Publication for Creative Developers & DevOps Enthusiasts. Medium’s largest DevOps publication.

Vikram Shinde

Written by

Cloud & DevOps Enthusiast. My old blogs can be found at below links http://vikshinde.blogspot.com/

FAUN

FAUN

The Must-Read Publication for Creative Developers & DevOps Enthusiasts. Medium’s largest DevOps publication.

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store