Getting started with Terraform — PART 1

Best practices for using Terraform in the real world.

What is Terraform?

Terraform is an open source “Infrastructure as Code” tool, created by HashiCorp.

A declarative coding tool, Terraform enables developers to use a high-level configuration language called HCL (HashiCorp Configuration Language) to describe the desired “end-state” cloud or on-premises infrastructure for running an application. It then generates a plan for reaching that end-state and executes the plan to provision the infrastructure.

Because Terraform uses a simple syntax, can provision infrastructure across multiple cloud and on-premises data centers, and can safely and efficiently re-provision infrastructure in response to configuration changes, it is currently one of the most popular infrastructure automation tools available. If your organization plans to deploy a hybrid cloud or multicloud environment, you’ll likely want or need to get to know Terraform.

Introduction - Terraform by HashiCorp

Why Infrastructure as Code (IaC)?

To better understand the advantages of Terraform, it helps to first understand the benefits of Infrastructure as Code (IaC). IaC allows developers to codify infrastructure in a way that makes provisioning automated, faster, and repeatable. It’s a key component of Agile and DevOps practices such as version control, continuous integration, and continuous deployment.

Infrastructure as code can help with the following:

• Improve speed: Automation is faster than manually navigating an interface when you need to deploy and/or connect resources.
• Improve reliability: If your infrastructure is large, it becomes easy to misconfigure a resource or provision services in the wrong order. With IaC, the resources are always provisioned and configured exactly as declared.
• Prevent configuration drift: Configuration drift occurs when the configuration that provisioned your environment no longer matches the actual environment. (See ‘Immutable infrastructure’ below.)
• Support experimentation, testing, and optimization: Because Infrastructure as Code makes provisioning new infrastructure so much faster and easier, you can make and test experimental changes without investing lots of time and resources; and if you like the results, you can quickly scale up the new infrastructure for production.

Why Terraform?

There are a few key reasons developers choose to use Terraform over other Infrastructure as Code tools:

• Open source: Terraform is backed by large communities of contributors who build plugins to the platform. Regardless of which cloud provider you use, it’s easy to find plugins, extensions, and professional support. This also means Terraform evolves quickly, with new benefits and improvements added consistently.
• Platform agnostic: Meaning you can use it with any cloud services provider. Most other IaC tools are designed to work with single cloud provider.
• Immutable infrastructure: Most Infrastructure as Code tools create mutable infrastructure, meaning the infrastructure can change to accommodate changes such as a middleware upgrade or new storage server. The danger with mutable infrastructure is configuration drift — as the changes pile up, the actual provisioning of different servers or other infrastructure elements ‘drifts’ further from the original configuration, making bugs or performance issues difficult to diagnose and correct. Terraform provisions immutable infrastructure, which means that with each change to the environment, the current configuration is replaced with a new one that accounts for the change, and the infrastructure is reprovisioned. Even better, previous configurations can be retained as versions to enable rollbacks if necessary or desired.

Terraform vs. Other Software

Procedural vs Declarative

Chef and Ansible encourage a procedural style where you write code that specifies, step-by-step, how to to achieve some desired end state. Terraform, CloudFormation, SaltStack, and Puppet all encourage a more declarative style where you write code that specifies your desired end state, and the IAC tool itself is responsible for figuring out how to achieve that state

For example, let’s say you wanted to deploy 10 servers (“EC2 Instances” in AWS lingo) to run v1 of an app. Here is a simplified example of an Ansible template that does this with a procedural approach:

- ec2:
    count: 10
    image: ami-v1    
    instance_type: t2.micro

And here is a simplified example of a Terraform template that does the same thing using a declarative approach:

resource "aws_instance" "example" {
  count         = 10
  ami           = "ami-v1"
  instance_type = "t2.micro"
}

Now at the surface, these two approaches may look similar, and they will produce similar results. The interesting thing is what happens when you want to make a change.

For example, imagine traffic has gone up and you want to increase the number of servers to 15. With Ansible, the procedural code you wrote earlier is no longer useful; if you just updated the number of servers to 15 and reran that code, it would deploy 15 new servers, giving you 25 total!

- ec2:
    count: 5
    image: ami-v1    
    instance_type: t2.micro

With declarative code, since all you do is declare the end state you want, and Terraform figures out how to get to that end state, Terraform will also be aware of any state it created in the past. Therefore, to deploy 5 more servers, all you have to do is go back to the same Terraform template and update the count from 10 to 15:

resource "aws_instance" "example" {
  count         = 15
  ami           = "ami-v1"
  instance_type = "t2.micro"
}

If you executed this template, Terraform would realize it had already created 10 servers and therefore that all it needed to do was create 5 new servers. In fact, before running this template, you can use Terraform’s plan command to preview what changes it would make:

$ terraform plan+ aws_instance.example.11
    ami:                      "ami-v1"
    instance_type:            "t2.micro"+ aws_instance.example.12
    ami:                      "ami-v1"
    instance_type:            "t2.micro"+ aws_instance.example.13
    ami:                      "ami-v1"
    instance_type:            "t2.micro"+ aws_instance.example.14
    ami:                      "ami-v1"
    instance_type:            "t2.micro"+ aws_instance.example.15
    ami:                      "ami-v1"
    instance_type:            "t2.micro"Plan: 5 to add, 0 to change, 0 to destroy.

Installing Terraform

Terraform must first be installed on your machine. Terraform is distributed as a binary package for all supported platforms and architectures.

To install Terraform, find the appropriate package for your system and download it. Terraform is packaged as a zip archive.

wget https://releases.hashicorp.com/terraform/0.12.24/terraform_0.12.24_linux_amd64.zip

After downloading, unzip the file on your directory, and add PATH in Bach_Profile for environment variable.

vim ~/.bash_profile
export Terraform=/home/salehsoft/dev
export PATH=$Terraform:$PATH

save file and run :

source ~/.bash_profile

Verifying the Installation

After installing Terraform, verify the installation worked by opening a new terminal session and checking that terraform is available. By executing terraform you should see help output similar to this:

If you get an error that terraform could not be found, your PATH environment variable was not set up properly. Please go back and ensure that your PATH variable contains the directory where Terraform was installed.

Configuration

The set of files used to describe infrastructure in Terraform is known as a Terraform configuration. You’ll write your first configuration now to launch a single AWS EC2 instance.

A configuration should be in its own directory. Create a directory for the new configuration.

$ mkdir learn-terraform-aws-instance

Change into the directory.

$ cd learn-terraform-aws-instance

Create a file for the configuration code.

$ touch example.tf

The format of the configuration files is documented here.

Paste the configuration below into example.tf and save it. Later in the guide when you run Terraform, it will load all files in the working directory that end in .tf.

provider "aws" {
   profile    = "default"   region     = "us-east-1"
 }
  resource "aws_instance" "example" {
   ami           = "ami-2757f631"
   instance_type = "t2.micro"
 }

Profile :

It refers to the AWS Config File in ~/.aws/credentials on MacOS and Linux or %UserProfile%\.aws\credentials on a Windows system. It is HashiCorp recommended practice that credentials never be hardcoded into *.tf configuration files. We are explicitly defining the default AWS config profile here to illustrate how Terraform accesses sensitive credentials.

Providers :

The provider block is used to configure the named provider, in our case "aws". A provider is responsible for creating and managing resources. A provider is a plugin that Terraform uses to translate the API interactions with the service. A provider is responsible for understanding API interactions and exposing resources. Because Terraform can interact with any API, almost any infrastructure type can be represented as a resource in Terraform.

Multiple provider blocks can exist if a Terraform configuration manages resources from different providers. You can even use multiple providers together. For example you could pass the ID of an AWS instance to a monitoring resource from DataDog.

Resources

The resource block defines a piece of infrastructure. A resource might be a physical component such as an EC2 instance, or it can be a logical resource such as a Heroku application.

The resource block has two strings before the block: the resource type and the resource name. In the example, the resource type is “aws_instance” and the name is “example.” The prefix of the type maps to the provider. In our case “aws_instance” automatically tells Terraform that it is managed by the “aws” provider.

Initialization

The first command to run for a new configuration — or after checking out an existing configuration from version control — is terraform init. Subsequent commands will use local settings and data that are initialized by terraform init.

Terraform uses a plugin-based architecture to support hundreds of infrastructure and service providers. The terraform init command downloads and installs providers used within the configuration, which in this case is the aws provider.

$ terraform init
 Initializing the backend...
  Initializing provider plugins...
 - Checking for available provider plugins...
 - Downloading plugin for provider "aws" (terraform-providers/aws) 2.10.0...
  The following providers do not have any version constraints in configuration, so the latest version was installed.  To prevent automatic upgrades to new major versions that may contain breaking changes, it is recommended to add version = "..."
 constraints to the corresponding provider blocks in configuration, with the constraint strings suggested below.
  * provider.aws: version = "~> 2.10"
  Terraform has been successfully initialized!
  You may now begin working with Terraform.
 Try running "terraform plan" to see any changes that are required for your infrastructure. All Terraform commands should now work.
  If you ever set or change modules or backend configuration for Terraform, rerun this command to reinitialize your working directory. If you forget, other commands will detect it and remind you to do so if necessary.

Apply Changes

In the same directory as the example.tf file you created, run terraform apply. You should see output similar to below, though we've truncated some of the output to save space.

$ terraform apply
 # ...
  An execution plan has been generated and is shown below.
 Resource actions are indicated with the following symbols:
   + create
  Terraform will perform the following actions:
    # aws_instance.example will be created
   + resource "aws_instance" "example" {
       + ami                          = "ami-2757f631"
       + arn                          = (known after apply)
       + associate_public_ip_address  = (known after apply)
...
...

This output shows the execution plan, describing which actions Terraform will take in order to change real infrastructure to match the configuration. The output format is similar to the diff format generated by tools such as Git. The output has a + next to aws_instance.example, meaning that Terraform will create this resource. Beneath that, it shows the attributes that will be set. When the value displayed is (known after apply), it means that the value won't be known until the resource is created.

If terraform apply failed with an error, read the error message and fix the error that occurred. At this stage, it is likely to be a syntax error in the configuration.

If the plan was created successfully, Terraform will now pause and wait for approval before proceeding. If anything in the plan seems incorrect or dangerous, it is safe to abort here with no changes made to your infrastructure. In this case the plan looks acceptable, so type yes at the confirmation prompt to proceed.

Executing the plan will take a few minutes since Terraform waits for the EC2 instance to become available.

# ...
 aws_instance.example: Creating...
 aws_instance.example: Still creating... [10s elapsed]   aws_instance.example: Creation complete after 1m50s [id=i-0bbf06244e44211d1]
  Apply complete! Resources: 1 added, 0 changed, 0 destroyed.

After this, Terraform is all done! You can go to the EC2 console to see the created EC2 instance. (Make sure you’re looking at the same region that was configured in the provider configuration!)

Terraform also wrote some data into the terraform.tfstate file. This state file is extremely important; it keeps track of the IDs of created resources so that Terraform knows what it is managing. This file must be saved and distributed to anyone who might run Terraform. It is generally recommended to setup remote state when working with Terraform, to share the state automatically, but this is not necessary for simple situations like this Getting Started guide.

You can inspect the current state using terraform show.

$ terraform show
# aws_instance.example:
resource "aws_instance" "example" {
    ami                          = "ami-2757f631"
    arn                          = "arn:aws:ec2:us-east-1:130490850807:instance/i-0bbf06244e44211d1"
    associate_public_ip_address  = true
    availability_zone            = "us-east-1c"
    cpu_core_count               = 1
    cpu_threads_per_core         = 1
    disable_api_termination      = false
    ebs_optimized                = false
    get_password_data            = false
    id                           = "i-0bbf06244e44211d1"
    instance_state               = "running"
    instance_type                = "t2.micro"
    ipv6_address_count           = 0
    ipv6_addresses               = []
    monitoring                   = false
    primary_network_interface_id = "eni-0f1ce5bdae258b015"
    private_dns                  = "ip-172-31-61-141.ec2.internal"
    private_ip                   = "172.31.61.141"
    public_dns                   = "ec2-54-166-19-244.compute-1.amazonaws.com"
    public_ip                    = "54.166.19.244"
    security_groups              = [
        "default",
    ]
    source_dest_check            = true
    subnet_id                    = "subnet-1facdf35"
    tenancy                      = "default"
    volume_tags                  = {}
    vpc_security_group_ids       = [
        "sg-5255f429",
    ]

    credit_specification {
        cpu_credits = "standard"
    }

    root_block_device {
        delete_on_termination = true
        iops                  = 100
        volume_id             = "vol-0079e485d9e28a8e5"
        volume_size           = 8
        volume_type           = "gp2"
    }
}

You can see that by creating our resource, we’ve also gathered a lot of information about it. These values can actually be referenced to configure other resources or outputs, which will be covered later in the getting started guide.

Manually Managing State

Terraform has a built in command called terraform state which is used for advanced state management. In cases where a user would need to modify the state file by finding resources in the terraform.tfstate file with terraform state list. This will give us a list of resources as addresses and resource IDs that we can then modify.

For more information about the terraform state command and subcommands for moving or removing resources from state, see the CLI state command documentation. This is outside the core Terraform workflow, but is worth noting as you learn how state is managed.

»Provisioning

The EC2 instance we launched at this point is based on the AMI given, but has no additional software installed. If you’re running an image-based infrastructure (perhaps creating images with Packer), then this is all you need.

However, many infrastructures still require some sort of initialization or software provisioning step. Terraform supports provisioners, which we’ll cover a little bit later in the getting started guide, in order to do this.

Troubleshooting

• If you use a region other than us-east-1, please use an AMI specific to that region as AMI IDs are region specific.
• If you do not have a default VPC in your AWS account in the us-east-1 region, create a new VPC in your VPC Dashboard in AWS. You will also need to associate a subnet and security group to that VPC. In your Terraform configuration, uncomment and modify the following two lines to your configuration for the rest of this track: vpc_security_group_ids (set as an array) and subnet_id with the corresponding information you just created. For more information, review this document from AWS on working with VPCs.

Destroy Infrastructure

We’ve now seen how to build and change infrastructure. Before we move on to creating multiple resources and showing resource dependencies, we’re going to go over how to completely destroy the Terraform-managed infrastructure.

Destroying your infrastructure is a rare event in production environments. But if you’re using Terraform to spin up multiple environments such as development, test, QA environments, then destroying is a useful action.

The terraform destroy command terminates resources defined in your Terraform configuration. This command is the reverse of terraform apply in that it terminates all the resources specified by the configuration. It does not destroy resources running elsewhere that are not described in the current configuration.

$ terraform destroy
  # ...
   # aws_instance.example will be destroyed
   - resource "aws_instance" "example" {
       - ami                          = "ami-b374d5a5" -> null
 # ...

The - prefix indicates that the instance will be destroyed. As with apply, Terraform shows its execution plan and waits for approval before making any changes.

Answer yes to execute this plan and destroy the infrastructure.

# ...
 aws_instance.example: Destroying... [id=i-0589469dd150b453b]  Destroy complete! Resources: 1 destroyed.
 # ...

Just like with apply, Terraform determines the order in which things must be destroyed. In this case there was only one resource, so no ordering was necessary. In more complicated cases with multiple resources, Terraform will destroy them in a suitable order to respect dependencies, as we'll see later in this guide.

To be continued …