Getting Started with Terraform and Digital Ocean

With the widespread of agile methodology, microservices, cloud services and continuous delivery among technology and non-technology companies, many have come to realize the need to build and secure their infrastructure as code and not in the head of any team member. Gone are days when only one geek in a company knows how to configure and spin up resources when the need arises because such situation leaves many companies at the mercy of these key men when they fall sick, go on vacation or move to a new company. Infrastructure as Code (IAC) is the only remedy to minimizing the challenges of shipping, rapidly iterating, and securing software applications and many fortune 500 companies have realized this. For those who are yet to, what are you still waiting for?

If you’re a devops engineer or just getting started, one of the first tasks you executed followed these lines below;

As expected, you’ll launch aws console and spend about 3 + minutes selecting various elements that is needed to spin up the elastic beanstalk application and some more time creating the two EC2 instances one after the other. Afterwards, you’ll pray that you selected the correct versions, security groups, vpc and availability zones correctly. Once the instances are created successfully without reporting any issue and the environment health is ok , there's this "I'm proud of myself" feeling that radiates your mind.

Should you really be proud? When you can spend less time with higher certainty had you built your Infrastructure As Code.

# Why Infrastructure as Code (IAC) ?

You receive that unpleasant slack notification that a server is unreachable. You follow your usual quick fix routines, e.g running eb logs or ls -l /var/log, only to find out that the server has actually crashed. You freeze! Immediately, you get a reminiscence of the hustle that you had to go through while trying to configure that server. You try to recall every component you had installed on the crashed server, plus their versions. It strikes you that you cannot even recall the order in which everything was installed, along with other nitty gritty. You request the ground to open up and swallow you, because your users have already started tweeting screenshot of your crashed website and you're not confident it will be up soon.

Instead of clicking around a web UI or SSHing to a server and manually executing commands, the idea behind IAC is to write code to define, provision, and manage your infrastructure. Some of the benefits are;

  1. Eliminate Fear of accidental misconfiguration.
  2. Store those source files and entire infrstsructure in version control and captured in commit logs. This can be used to debug problems, and if necessary, roll back to older versions.
  3. Work continues if the sysadmin goes on Vacation.
  4. Mitigates some failures.
  5. Empower developers and operators to create infrastructure with shared templates, thereby keeping developers happy with fast deployments.
  6. Devops will always say YES!! to all infrastructure requests from engineers.
  7. Distribution of knowledge.
  8. Eliminates key-man syndrome.
  9. Validate each infrastructure change through code reviews and automated tests.

There are numerous infrastructure build tools out there, however in this tutorial we’ll explore the nitty gritty of Terraform which is one of the best open source tool that allows you to safely and predictably create, change, and improve production infrastructure using a language that is human and machine friendly. Interestingly, it was built by Hashicorp the famous makers of Vagrant.

Why terraform ?

I heard the word infrastructure-as-code for the first time during a meetup few months after college. As an inquisitive person, I headed to the repository of knowledge to know what this buzz word really meant. I read and read until I stumbled on a blog that provided a list of IAC tools. One underlying characteristics of all of the tools was that they were well documented, both in terms of official documentation and community resources such as blog posts, Quora and StackOverflow questions. So how do you decide on which tool to use for IAC.

Some reasons why Terraform is second to none amongst IAC tools are listed below. As with all technology decisions, it’s a question of trade-offs, priorities and features.

  1. Configuration Management vs Orchestration: Some IAC tools like Chef, Puppet, Ansible, and SaltStack are all configuration management tools, which means they are designed to install and manage software on existing servers. CloudFormation by Amazon and Terraform by Hashicorp are orchestration tools, which means that they are designed to provision the servers themselves, leaving the job of configuring those servers to other tools. However, some tools perform a bit of orchestration and a bit of configuration management. But the focus on configuration management or orchestration means that some of the tools are going to be a better fit for certain types of tasks. Interestingly, by using Docker or Packer together with Terraform, the burden of configuration management is resolved which makes it the best fit for your infrastructure.
  2. Procedural vs Declarative: Some IAC tools like Chef and Ansible encourage a procedural style where you write code that specifies, step-by-step, how to to achieve some desired end state. Terraform, CloudFormation, SaltStack, and Puppet all encourage a more declarative style where you write code that specifies your desired end state, and the IAC tool itself is responsible for figuring out how to achieve that state. Who really wants to define in infrastructure step by step ?
  3. Mutable vs Immutable Infrastructure: Immutable means unchanging over time or unable to be changed while mutable is the opposite. Configuration management tools such as Chef, Puppet, Ansible, and SaltStack typically default to a mutable infrastructure paradigm. For instance, if Chef is used to install a new version of MySQL, it’ll run the software update on your existing servers and the changes will happen in-place. Over time, as you apply more and more updates, each server builds up a unique history of changes. This often leads to a terminology known as configuration drift, where each server becomes slightly different than all the others, leading to subtle configuration bugs that are difficult to diagnose and nearly impossible to reproduce. With an Immutable infrastructure like Terraform, every change to the images created by Docker or Packer, is actually a deployment of a new server. This approach reduces the possibility of configuration drift bugs, makes it easier to know exactly what software is running on a server, and allows you to easily deploy any previous version of the software at any time. Version control systems, such as git, enjoy widespread use because they offer tremendous benefits of immutability which makes your code versionable, allowing rollback and roll forwards.

OVERVIEW OF TERRAFORM

The official Terraform Getting Started is a great place to learn about the individual elements of Terraform (i.e. resources, input variables, output variables, etc), so in this guide, we’re going to focus on how these elements can be combined to spin up resources that can run some web applications. In particular, we’ll create Digital Ocean Droplets, DNS Records and Domain Names. The infrastructure you’ll create in this example is a basic starting point for running scalable, highly-available web applications.

Terraform uses a configuration syntax called HashiCorp Configuration Language (HCL). The goal is to build a configuration language that is both human and machine friendly for use with command-line tools to run DevOps and server relates commands. Generally, it is recommended you use the HCL Terraform syntax. Therefore, in this tutorial, we’ll use the HCL syntax for our code samples.

This guide is written with Digital Ocean and Terraform Newbies in mind, so don’t worry if you haven’t used either one before. We’ll walk you through the entire process by covering the following;

  • Set up your Digital Ocean Account.
  • Generate Digital Ocean Personal Access Token.
  • Install Terraform.
  • Create a Digital Ocean Droplet.
  • Create a domain and record set.
  • Change Infrastructure.
  • Clean up.

Set up your Digital Ocean Account

Terraform can provision infrastructure across many cloud providers, including AWS, Azure, Google Cloud, Digital Ocean, Heroku as listed here. In this tutorial we’ll dwell more on how to use terraform on Digital Ocean because of its simplicity and robustness.

First of all, you’ll need a Digital Ocean account, so head over to Digital Ocean and Sign up for an account.

Once you complete the two field form, you’ll get an email containing a link to complete the signup process.

Click on the link to verify your email and setup your billing info. You can either add a credit card or use pay pal. If you need a virtual card that’s not tied to your bank account for security reasons, you can generate one using Barter. This guide explains the steps to follow for Amazon which can also be followed for Digital Ocean.

Once you’re done filling the billing info, we’re all set to start running some scripts to spin up resources.

Generate a Digital Ocean Personal Access Token

The personal access token is a very important and sensitive information that can be used to modify resources on Digital Ocean. To generate a new token, click on API in the top menu.

Next, In the Tokens section, click the Generate new token button:

You will be taken to the New Personal Access Token screen:

Here, provide the following information:

Your desired token name (for your own reference) Select the scope for this token (read or read/write)

You’ll have to check the write scope to allow terraform create and edit resources.

Then click the Generate Token button.

Your token will be generated and presented to you on your Personal Access Tokens page. The actual token is the long string of numbers and letters, under the name:

Be sure to record your personal access token now because for security reason It will not be shown again once the page is refreshed.

Install Terraform

For mac OSX users who use brew as a package manager, you can install terraform by simply running on your terminal; Else you can follow the guide here to install terraform. As at the time of writing this article the stable terraform version was 0.8.8 although the release of version 0.9 has been announced.

$ brew install terraform

After installation, confirm that terraform was installed by running;

$ terraform

You should see an output like this;

In order for Terraform to successfully make changes to your Digital Ocean account, you will need to set the Personal token as an environment variable on your machine and here’s how to do that; Terraform variables are prepended by TF_VAR_ which makes them easy to spot. If we refer to a variable called digitalocean_token inside Terraform, we need to name it TF_VAR_digitalocean_token using the EXPORT command:

$ export TF_VAR_digitalocean_token= xxxxxxxxx //your token goes here
Do not hardcode your token(s) into programs use environmental variables instead. If a token becomes compromised, delete it to revoke that token’s access.

Create a Digital Ocean Droplet

Terraform code is written in a language called HCL in files with the extension “.tf”. It is a declarative language, so your goal is to describe the infrastructure you want, and Terraform will figure out how to create it. Terraform loads all .tf files in any directory where a terraform command is called. So ensure that there are no other *.tf files in your working directory.

The first step to using Terraform is to configure the provider(s) you want to use. Create a file called digital_ocean_create_droplet.tf or whatever name you want and put the following code in it:

provider "digitalocean" { token = "${var.digitalocean_token}" // referencing the token you exported earlier using var }

The command above tells Terraform that you are going to be using the Digital Ocean provider and declare the token, which we exported earlier to our machine.

For each Terraform provider, there are different resources you can create like servers, databases, domain Names, domain name records etc. In this tutorial we'll create a single digital ocean server which is called a DROPLET in digital ocean lingo. To create a droplet, add the following lines to your digital_ocean_create_droplet.tf file;

# Create a new Web Droplet running ubuntu in the FRA1 region resource "digitalocean_droplet" "terraform" { image = "ubuntu-14-04-x64" name = "terrform-droplet" region = "fra1" size = "1gb" }

Each resource specifies a type (in this case, “digitalocean_droplet”), a name (in this case “terraform”) which is used as an identifier within the Terraform code, and a set of configuration parameters specific to the resource.

Now, check the correctnes of your code by running terraform validate

$ terraform validate

This is one of those “no news is good news” output results. If you don’t see anything, the validation was successful.

Next, we show the planned results of our Terraform script using the terraform plan command. This is definitely one of the features that differentiates terraform from the rest. It's very important to review the plan to see what Terraform will do before actually doing it. This is a great way to sanity check your changes before unleashing them onto the world.

Open your terminal and navigate into the folder where you created digital_ocean_create_droplet.tf. Now run the terraform plan command: Your plan output should look like this;

Careful review of your plan will help you mitigate what happened to Amazon S3 even with an established playbook.

You will see some information on the screen about using an -out parameter. This is another feature of the terraform plan directive for capturing a point-in-time view of the environment and keeping a persistent version on disk or in memory. For now, let’s move on to the launch our plan.

After reviewing your plan, building this infrastructure is just a terraform apply away and a terraform show to provide human-readable output from the state or plan file. After executing the plan commands, terraform stores 2 files (terraform.tfstate.backup and terraform.tfstate) in the directory which is used to track the state of the resources created. These maintain a copy of the state of our Terraform environment and will also come into play as we dive in further with the Terraform goodness.

Congrats, you’ve just created a droplet with Terraform! To verify this, you can login to Digital Ocean Console, and you’ll see something like this:

Once you confirm that the droplet was created on the web console, go back to your digital_ocean_create_droplet.tf and let's create a domain name.

Create a Domain Name and a DNS record

To create a new domain, the following variables are required.

name — (Required) The name of the domain ip_address — (Required) The IP address of the domain. This IP is used to create an initial A record for the domain. This can either be an existing IP address or you use digitalocean_droplet.terraform.ipv4_address to fetch the ip address of the droplet we created earlier. It is required upstream by the DigitalOcean API.

Now open your digital_ocean_create_droplet.tf file and add the following lines;

# Create a new domain resource "digitalocean_domain" "terraform" { name = "coundigital.com" ip_address = "${digitalocean_droplet.terraform.ipv4_address}" #the ip address of the droplet created earlier }

If you have an existing droplet on digital ocean or an IP elsewhere you want the new domain to point to, then your code should look like this;

# Create a new domain resource "digitalocean_domain" "terraform" { name = "coundigital.com" ip_address = "46.101.205.36" #if you already have a digital ocean droplet/server ip }

Now Add more lines of code to create a CNAME record like so;

# Add a record to the domain resource "digitalocean_record" "scotch" { domain = "${digitalocean_domain.terraform.name}" // fetching the domain name created above. type = "CNAME" name = "terrraform-scotch" value = "skyrunonline.com." # Take note of the point after the domain name. pointing to the ip of an existing droplet on Digital Ocean }

Once you have updated your config, run terraform plan to confirm changes. Once the update is confirmed, run terraform apply and watch terraform create the new domain and cname record. Once the command is done executing, go to your digital ocean domain console to view the domain.

Click on the domain you just created to view the records;

When you must have updated all examples, your digital_ocean_create_droplet.tf file should look like this;

# Configure the DigitalOcean Provider provider "digitalocean" { token = "${var.digitalocean_token}" } # Create a new domain resource "digitalocean_domain" "terraform" { name = "coundigital.com" ip_address = "${digitalocean_droplet.terraform.ipv4_address}" #the ip address of the droplet created earlier # Add a record to the domain resource "digitalocean_record" "scotch" { domain = "${digitalocean_domain.terraform.name}" type = "CNAME" name = "terrraform-scotch" value = "${digitalocean_domain.terraform.name}." }

Update your Infrastructure

The immutable nature of terraform plays a key role during infrastruture updates. Values of Infrastructures are continuously evolving and changing and terraform is very much aware of these changes and hence it was built to help manage and enact those changes. As you change Terraform configurations, terraform builds an execution plan that only modifies what is necessary to reach your desired state. This is what immutability bring to the table

Let’s assume that after creating the 1GB terraform droplet initially, the requests on the server rose beyond expectation days later and you’ve been summoned by your boss to increase the ram size to 2GB. Simply edit your code to reflect this change and watch how terraform helps you carry out the change. As usual open digital_ocean_create_droplet.tf file and make the change like so;

# Create a new Web Droplet running ubuntu in the FRA1 region resource "digitalocean_droplet" "terraform" { image = "ubuntu-14-04-x64" name = "terrform-droplet-change" region = "fra1" size = "2gb" // changed to 2GB }

Save the file and head over to your terminal and run terraform plan

Notice I also changed the droplet name from terrform-droplet to terrform-droplet-change and terraform also captured it. By using Terraform to change infrastructure, you can version control not only your configurations but also your state so you can see how the infrastructure evolved over time.

Clean up and Final Words

When you’re done experimenting with Terraform, it’s a good idea to remove all the resources you created so that Digital Ocean doesn’t charge you for them. Since Terraform keeps track of what resources you created, cleanup is a breeze. All you need to do is run the terraform destroy command. However, before you run terraform destroy see what will be destroyed by running;

terraform plan --destroy

Your output should look like this;

Once you type in “yes” and hit enter, terraform will build the dependency graph and delete all the resources in the right order, using as much parallelism as possible. In about a minute, your Digital Ocean account should be clean again.

In recap, we’ve been able to establish that Orchestration, Declarative Language Nature and Immutability Infrastructure as some advantages to choose trafform over other IAC tool. We also emphasized the need to always terraform plan before you terraform apply. Lastly, we cleaned up all the resources we created uisng terraform detroy.

That is my first and very simple example of using Terraform. My next tutorial will feature a more complex build in conjunction with some other configuration using the same DigitalOcean provider. Feel free to use the comments section for questions and suggestions.


Originally published at scotch.io.

Like what you read? Give Chidiebube Amos a round of applause.

From a quick cheer to a standing ovation, clap to show how much you enjoyed this story.