Infrastructure-as-Code: what is it and how can you use it?

The following text is a part of Kiwi.com’s internal DevOps handbook. We thought it could come in handy for others as well, so we decided to share it with the world here, like we’ve done before here and here.

What is Infrastructure-as-Code?

Infrastructure-as-Code is a process for managing IT infrastructures.

It frees up system administrators from needing to perform numerous administrative tasks before reaching their intended goals.

It also allows admins to create a descriptive model of the infrastructure configuration that they have in mind, which then will be realised by a software tool.

What are the benefits of Infrastructure-as-Code?

Imagine starting a new cloud-based web service. In order to launch this service, you would first need to create a virtual machine on one of the main cloud computing providers, such as Google Cloud Platform or Amazon Web Services. Next, you would need to go through various administrative consoles to configure the virtual machine and expose it to the public internet.

You can adequately launch a service in this way if you do not expect any changes. But what if you need to scale your infrastructure or expand the capabilities of your service? Maintaining your infrastructure using administrative consoles would then become a very tedious endeavour.

Having a descriptive model of the infrastructure not only allows for quick editing of the infrastructure but also enables its versioning and reusability. You can have an incremental history of what has changed, but also duplicate the model without the minute inconsistencies that might creep in with manual administration.

How do I use Infrastructure-as-Code?

In this guide, we will use Terraform to demonstrate the Infrastructure-as-Code workflow. It’s a software tool that enables you to define and provision server infrastructure using a high-level scripting language.

Analogous to languages like HTML and SQL, Terraform is a declarative language that allows you to describe the infrastructure topology and its desired state.

Once defined, Terraform will work out how to achieve this state. Subsequently, it will execute the necessary steps by itself.

This approach to infrastructure provisioning provides you with a workflow for creating and managing the high-level resources of your infrastructure, such as Virtual Private Clouds, Virtual Machines, and Security Group Rules, while alleviating you from setting up each of these resources manually.

This functionality is provided by so-called provider plugins. Terraform has out-of-the-box plugins for providers like Cloudflare or Google Cloud Platform, but there are also unofficial plugins available for providers such as Okta. You can install these plugins by copying the binary to the root directory of your Terraform project.

Quick start: Create a “Hello world” web server on GCP using Terraform

This tutorial will demonstrate Terraform’s key functionalities by guiding you through the steps for creating a Virtual Machine on Google Cloud Platform, and a Python Flask app that’ll serve a “Hello World” page on the IP address of your instance. The step-by-step instructions are based on this tutorial from Google.

Please follow the steps below to create your “Hello World” web server on GCP:

1. Creating a Google Cloud Platform project

First, we must create a new GCP project and fill out the project details.

a. Go to your Google Cloud Platform console.

b. Click on the Select a project button in the blue top bar of the console, then click on the NEW PROJECT button in the dialog box that appears.

c. Fill out the project details on the New Project page. Don’t forget to copy down the Project ID. You will need it to complete your main Terraform configuration file.

2. Obtaining the credentials of your GCP project

Next, we need to obtain a JSON file with the project credentials. Terraform needs these credentials for accessing your GCP project and managing your resources.

a. Go to the create service account key page.

b. Select the default service account or create a new one.

c. Select JSON as the key type and click on Create. This will download a JSON file to the download folder of your browser. Please move this file to a secure place. You will need it to complete the next steps.

3. Setting up your Terraform project directory

Now, you will set up your Terraform project. You will create your project’s main configuration file. Once Terraform will initialise your project, it will download the necessary plugins to configure your GCP infrastructure.

In the case that you need to use an unofficial provider plugin, such as this Okta plugin, you’ll need to copy the binary to the root directory of your Terraform project.

a. Create a new directory for your project and create an empty main.tf file.

b. Copy the code block below into this file.

// Configure the Google Cloud provider
  provider "google" {
  credentials = "${file("CREDENTIALS_FILE.json")}"
  project     = "flask-app-211918"
  region      = "us-west1"
  }

c. Change the project property from flask-app211918 to the Project ID that you obtained in Step 1c.

d. Rename the filename in the credentials property, currently CREDENTIALS_FILE.json, to the name of the JSON file that you downloaded in Step 2c.

e. In your terminal application, go to your project folder, and enter the command terraform init. This command will download a plugin for linking up your Terraform project with GCP and build the .terraformdirectory.

f. You will see the following output if Terraform processed your main.tf file successfully:

Initializing provider plugins...
    - Checking for available provider plugins on https://releases.hashicorp.com...
    - Downloading plugin for provider "google" (1.16.2)...

    The following providers do not have any version constraints in configuration,
    so the latest version was installed.

    To prevent automatic upgrades to new major versions that may contain breaking
    changes, it is recommended to add version = "..." constraints to the
    corresponding provider blocks in configuration, with the constraint strings
    suggested below.

    * provider.google: version = "~> 1.16"

    Terraform has been successfully initialized!

4. Configuring your Google Compute Instance resources

Next, you will add two resources, random_id and google_compute_instance, to your main.tf file, including four properties for your google_compute_instance resource, namely book_disk, ,etadata_startup_script, and network_interface.

Once Terraform initialises these additions, it will download the random_id plugin. This plugin creates random instance names that comply with GCP’s naming requirements.

a. Add the below code block to your main.tf file:

// Terraform plugin for creating random ids
    resource "random_id" "instance_id" {
    byte_length = 8
    }

    // A single Google Cloud Engine instance
    resource "google_compute_instance" "default" {
    name         = "flask-vm-${random_id.instance_id.hex}"
    machine_type = "f1-micro"
    zone         = "us-west1-a"

    boot_disk {
      initialize_params {
        image = "debian-cloud/debian-9"
      }
    }

    // Make sure flask is installed on all new instances for later steps
    metadata_startup_script = "sudo apt-get update; sudo apt-get install -yq build-essential python-pip rsync; pip install flask"

    network_interface {
      network = "default"

      access_config {
        // Include this section to give the VM an external ip address
      }
    }
    }

b. Run terraform init. This command will download and install the random_id Terraform plugin.

c. Run terraform plan. This command will show a preview of what Terraform will create in your GCP project. You will see the following output:

An execution plan has been generated and is shown below.
    Resource actions are indicated with the following symbols:
      + create

    Terraform will perform the following actions:

      + google_compute_instance.default
          id:                                                  [computed]
    ...

      + random_id.instance_id
          id:                                                  [computed]
    ...


    Plan: 2 to add, 0 to change, 0 to destroy.

d. Run terraform apply. This command will call the Google Cloud Platform APIs to set up the new instance.

e. Check the VM Instances page in the Google Cloud Platform UI. The new instance will be there.

5. Enabling SSH access to your instance

In order to create a web server on your newly created virtual machine, you will need to establish remote access to it. The below steps guide you through the process of enabling SSH access to your instance. Here you can find more information on managing SSH keys.

a. Add the metadata property in the code block below to the google_compute_instance resource of your main.tf file.

resource "google_compute_instance" "default" {
    ...
    metadata {
      sshKeys = "INSERT_USERNAME:${file("~/.ssh/id_rsa.pub")}"
    }
    }

b. Replace /.ssh/id_rsa.pub with the location of your Google Cloud Engine instance’s public SSH key.

c. Replace INSERT_USERNAME with your username.

d. Run terraform plan to verify the output.

e. Run terraform apply to apply the changes. You will see the following output:

An execution plan has been generated and is shown below.
    Resource actions are indicated with the following symbols:
      ~ update in-place

    Terraform will perform the following actions:

      ~ google_compute_instance.default
          metadata.%:       "0" => "1"
    ...

    Apply complete! Resources: 0 added, 1 changed, 0 destroyed.

6. Extracting the external IP address of your instance

Next, you’ll need to expose the external IP address of your instance.

a. Add the below code block to your main.tf file.

// A variable for extracting the external ip of the instance
    output "ip" {
    value = "${google_compute_instance.default.network_interface.0.access_config.0.nat_ip}"
    }

b. Run terraform apply followed by terraform output ip to return the instance’s external IP address.

c. SSH to that IP address to validate that everything is set up correctly.

ssh `terraform output ip`

7. Running a web server on your GCP VM instance

Now you will set up your web server, which is a Python app that uses the Flask framework to respond to HTTP requests.

a. Go to your VM instance and create an empty file named appy.py

b. Copy the below code block into this file:

from flask import Flask
    app = Flask(__name__)

    @app.route('/')
    def hello_world():
      return 'Hello World!'

    app.run(host='0.0.0.0')

c. Run python.app.py

d. Run curl localhost:5000 in a separate SSH instance to confirm that your web server is running.

8. Opening port 5000 on your instance

As Flask serves traffic on localhost:5000 by default, you’ll need to expose this port on your instance by adding a firewall rule to your main.tffile.

a. Go to your VM instance and create an empty file named app.py

b. Add the below code block to your main.tf file:

resource "google_compute_firewall" "default" {
    name    = "flask-app-firewall"
    network = "default"

    allow {
      protocol = "tcp"
      ports    = ["5000"]
    }
    }

c. Run terraform plan and then terraform apply to create the firewall rule.

d. Go to your browser and enter your instance’s IP address and port 5000. You should be able to see the output of your web server.

Next steps

Now you’ve learned how easy it is to create a basic cloud infrastructure using Terraform’s init, plan, and apply commands. If you’re curious to find out more about this topic, please join our Summer Camp for Senior Devs in Barcelona. You’ll have the chance to meet our Platform engineers and build cool projects under their guidance.

Do you enjoy the art of building cloud infrastructures? Check our open positions and join our Platform Team in making travel better.