Spring Boot CI/CD on Kubernetes using Terraform, Ansible and GitHub: Part 3

Part 3: Automatic creation of the cloud infrastructure using Terraform

This is part of a series of articles that creates a project to implement automated provisioning of cloud infrastructure in order to deploy a Spring Boot application to a Kubernetes cluster using CI/CD. In this part we provision the Virtual Private Cloud (VPC) and Virtual Private Servers (VPS) using the Terraform Infrastructure as Code (IaC) tool.

Follow from the start — Introduction

In this article, we will create the Virtual Private Cloud (VPC) and Virtual Private Servers (VPCs) that will host our Kubernetes cluster. We will use HashiCorp’s Open Source Terraform application to set up these resources using Infrastructure as Code techniques.

Whilst HashiCorp provide a cloud based version of Terraform, we will use the local CLI version on our development machine. This was installed during part 2 of the series.

You can find the code for this part here:
https://github.com/MartinHodges/Quick-Queue-IaC/tree/part3

Infrastructure as Code

Infrastructure as Code (IaC) is a term that covers the creation of virtual infrastructure, normally in the cloud, using a declarative definition of what you want.

Imperative vs Declarative definitions

When talking about IaC, it is important to understand the difference between imperative and declarative definitions.

Imperative Definitions: As a Developer you are probably familiar with imperative definitions. This is when you write the instructions you want the computer to follow, such as using if, while and for statements.

Think of it like this. If you were inviting your friends round for a meal, using imperative definitions you would write the recipes for the food you were going to make and serve.

Declarative Definitions: This is where you specify the results of what you want and not how to get there. You leave it to the tooling to work out how to create what you want.

In the example above, you would create the menu for the meal and leave it to the kitchen staff to decide how to make and serve it.

As developers, we tend to think in an imperative way but, when creating Infrastructure as Code (IaC), we need to think in terms of declarative definitions. Effectively we will be creating a shopping list of what you want.

In this project we will be asking for the creation of a Virtual Private Cloud (VPC) and 3 Virtual Private Servers (VPSs).

Setup vs Provisioning

Now you understand what it means to create a declarative definition, you now need to understand the difference between the two steps required to get your cloud infrastructure up and running:

• Setup
• Provisioning

Setup: This is the process whereby you create the virtual infrastructure you need, such as networks, servers and load balancers.

Provisioning: This is the process whereby you configure the infrastructure you have set up by creating configurations, installing packages and setting up users etc.

In some descriptions, the term IaC only applies to the setup but I prefer to consider both parts as IaC as both are necessary to create infrastructure we can use. I’ll leave it to you to decide how you use the terms.

Whilst IaC tools can do both setup and provisioning, each has been primarily designed with one of them in mind. In this project we will use:

• Terraform to setup our infrastructure
• Ansible to provision our infrastructure

There are other tools on the market but these are free, open source solutions that are widely used and documented.

Managing State

Before we jump into the Terraform setup, we need to understand how state is managed.

With declarative definitions, it is important that the IaC tools can track the actual state and the desired state so that the tool can issue the required instructions to change the actual state to the desired state.

Terraform and Ansible use different techniques for tracking state. I like to think of the differences as shopping lists and trollies.

Tracking state as a shopping list

Imagine you are in the supermarket and you have to buy your groceries. You take a shopping list with you.

As you pickup items and put them in your trolly, you tick them off your list. This is quick and efficient as you only need keep your list in your hand and you can quickly scan the list for things you have missed.

The problem with using the shopping list as your current state, is that you never know if someone has taken anything out of your trolley or put something in while you were not looking (anyone with young kids will relate to this!).

Tracking state in the trolly

The alternative is to still use the shopping list but each time you go to pick up your item, you first search through your trolley to see if you still have everything you expect and nothing you don’t. This way you are absolutely certain about your grocery items in your trolly when you get to the checkout.

The problem with using the trolly as the current state, is that it is slow. Each time you want to make a change you have to look in the trolley and determine what needs to be changed. The advantage is that you take into account any changes that happened ‘behind the scenes’.

Terraform

Terraform uses the shopping list technique for tracking state. It keeps a list in a file on your development machine of the things that it thinks you have in your cloud account and makes adjustments based on that list. If you make any changes ‘behind the scenes’, things will go wrong.

Ansible

Unlike Terraform, Ansible uses the trolley to track state. Each time it runs, it actually takes a look into the VPS it is about to configure and checks to see what it needs to do.

This difference between the tools means that, whilst Terraform appears to run more quickly, it can create the wrong infrastructure if you dabble directly with the services in your cloud account. Ansible on the other hand, allows for your dabbling but takes longer to work.

Setting up the VPC and VPS infrastructure

We will now use Terraform to create our Virtual Private Cloud (VPC) and 3 Virtual Private Servers (VPSs) in that VPC.

In the introduction to this series, I discussed the fact that I am using the Australian Binary Lane service for my cloud provider. This provides a challenge when using Terraform but also gives us the opportunity to learn more about Terraform than we would if we used a well known, fully featured cloud provider such as AWS, Azure or Google cloud.

Binary Lane API

Whenever you want to create your cloud-based services using IaC, your tooling (ie: Terraform) will be using the API of your selected cloud provider to set up the resources you want. Like other cloud services, Binary Lane has a REST API for this purpose, documented here.

You gain access to this API once you have an account. Access is controlled through your account API keys. To create an API key, go here:

Developer API | BinaryLane Australia

Click + Create Token, give it a name (eg: quick queue) and take a note of the token created. This is only visible once. If you lose it you will need to regenerate it and update your configuration with the new key. Keep the key safe as it gives anyone with access to your key the ability to create and destroy your cloud services.

The Binary Lane API gives you access to the following resources:

• * Servers
• * Virtual Private Cloud
• Server management
• Data usage (there are charges for exceeding limits)
• Fail over IP addresses
• OS images
• SSH Key management
• Reference data
• OS Images
• Regions
• Sizes
• Load balancers
• DNS Management
• Others

* In these articles we will only be using the Server and Virtual Private Cloud endpoints.

How Terraform works

Terraform takes your declarative definition of what you want and then interfaces to the relevant APIs to set it up.

It keeps track of what it does and each time it runs it compares what you are asking for (the target state) against what it is tracking (the actual state). As it uses the ‘shopping list’ approach, any changes you make directly through the Binary Lane user interface, will not be known to Terraform and this can result in errors that can be hard to fix up.

Once Terraform successfully makes a change, it modifies the actual state it is holding in its ‘shopping list’.

If an error occurs, Terraform will tell you that it is possible that the change was actually made and that it does not know. This means that its ‘actual state’ may be incorrect leading to more errors.

I have hit this problem several times during the creation of this series of articles and found the easiest way is to delete everything and start again. The beauty of IaC is that this does not take long and very few commands.

Note: if things do go wrong, Terraform can end up creating more and more resources. If things go wrong, check your Binary Lane account for duplicates as these will increase the fees you pay to Binary Lane.

Providers

You may be wondering how Terraform can take into account so many different cloud services with so many different services.

Terraform uses plugins called Providers. A Provider takes service specific definitions of what you want (the target state) and interfaces to the relevant service API to give you what you asked for.

If you have ever worked with AWS, you will know there are hundreds of services and people have built hundreds of Providers to set them up.

For Binary Lane it is a different matter. There are no Binary Lane specific Providers to use.

Binary Lane Provider

Whilst AWS, Google Cloud and Azure all have Provisioners for all their services, Binary Lane does not currently have any (may be this is a project you would be interested in?).

As mentioned earlier, Binary Lane does have a REST API and there is a generic Provider that can manage bespoke REST APIs. It does require a little care and attention but the benefit is that you will learn more than simply using a well crafted AWS Provider.

We will be using this generic Provider, built and maintained by MasterCard as an open source project. More about this later but first a few other Terraform principles.

Provisioners

I think it is worth mentioning Provisioners at this point. Terraform supports a concept of Provisioners. These are plugins that can do the configuration of your infrastructure once it is set up.

There is a lot of commentary about Provisioners and the advice is not to use them and to use a tool such as Puppet, Chef or Ansible.

For this reason, I will not be talking about Provisioners in this series.

Terraform — folders and files

When you run Terraform, it reads the configuration from all files in the current folder. This includes:

• *.tf — contain a definition of the infrastructure to setup
• *.tfvars — define key-value pairs as inputs to the setup
• *.tftpl — template files that can be used to create result files
• *.tfstate — the actual state of the infrastructure according to Terraform
• .terraform — a folder where Terraform holds its project set up

Modules

When creating infrastructure, it is useful to be able to reuse a particular setup. For example, when creating multiple copies of of a VPS.

Terraform uses the concepts of Modules. A Module is like a subroutine that performs the set up of a particular item of infrastructure. It is parameterised so you can pass it inputs, such as the VPS name. It also has the ability to return results of the Module creation and set up.

In this project the Modules can be found as subfolders under themodules folder.

Kubernetes Cluster

Before we create the infrastructure, a quick description of Kubernetes (also known as K8s (K followed by 8 letters followed by s!).

Kubernetes can work with a single node and can scale to any number (that your budget allows).

In practice most examples include three, which is what we will do here.

• A master node that manages the cluster
• Worker node 1
• Worker node 2

The master node schedules your applications, which means it starts them, on the worker nodes using a selection algorithm which we will not go into here.

At the moment, it is sufficient to say we need 3 servers in the cloud for our example.

Configuring Terraform

For this project, we will create files that will setup:

• 1 Virtual Private Cloud
• 1 VPS Kubernetes master node
• 2 VPS Kubernetes worker nodes

A VPC is a Software Defined Network (SDN) that allows your cloud provider to create an isolated subnet for your servers. Servers in the VPC may be dual homed, which means they have an interface connected to your private subnet and an interface connected to the Internet with a public IP address. Your cloud provider allocates all public and private IP addresses.

If the VPS has a public Internet connection, I refer to it as a public VPS. It it does not, I refer to it as a private VPS.

Kubernetes requires that the master node is a 2 cpu node whereas the worker nodes can be 1 cpu nodes.

We will add all Terraform configuration files to the Quick-Queue-IaC/terraform project folder we created earlier. Use whatever editor you are most comfortable with.

We will start with the main.tf. Terraform files can be written in Hashicorp Configuration Language (HCL) or JSON. For these examples we will be using HCL.

The terraform/main.tf File

In this series of articles, I prefer to give you the whole file first and then describe it in sections.

First we will create terraform/main.tf in the Quick-Queue-IaC sub-project.

terraform {
required_version = ">= 1.5.5"
  required_providers {
    restapi = {
      source = "Mastercard/restapi"
    }
  }
}

module "qq_vpc" {
  source = "./modules/vpc"
  vpc_name = "qq_vpc"
}

output "qq_vpc" {
  description = "qq_vpc"
  value = module.qq_vpc.vpc_desc
}

module "qq_k8s_master" {
  source = "./modules/public_vps"
  vps_name = "qq-k8s-master"
  vps_flavour = "std-2vcpu"
  vpc_id = module.qq_vpc.vpc_desc.id
  ssh_key = var.ssh_key
}

output "qq_k8s_master" {
  description = "qq_k8s_master"
  value = module.qq_k8s_master.v4_ips
}

module "qq_k8s_node_1" {
  source = "./modules/public_vps"
  vps_name = "qq-k8s-node-1"
  vps_flavour = "std-1vcpu"
  vpc_id = module.qq_vpc.vpc_desc.id
  ssh_key = var.ssh_key
}

output "qq_k8s_node_1" {
  description = "qq_k8s_node_1"
  value = module.qq_k8s_node_1.v4_ips
}

module "qq_k8s_node_2" {
  source = "./modules/public_vps"
  vps_name = "qq-k8s-node-2"
  vps_flavour = "std-1vcpu"
  vpc_id = module.qq_vpc.vpc_desc.id
  ssh_key = var.ssh_key
}

output "qq_k8s_node_2" {
  description = "qq-k8s_node_2"
  value = module.qq_k8s_node_2.v4_ips
}

resource "local_file" "inventory" {
  filename = "../ansible/inventory"
  content = templatefile("ansible_inventory.tftpl", {
    master_ip = module.qq_k8s_master.v4_ips.public
    node_ips = [
      module.qq_k8s_node_1.v4_ips.public,
      module.qq_k8s_node_2.v4_ips.public
    ]
  })
}

This is the main configuration file that will create our VPC and our 3 VPC nodes.

terraform {
required_version = ">= 1.5.5"
  required_providers {
    restapi = {
      source = "Mastercard/restapi"
    }
  }
}

This first section defines the version of Terraform required (≥ 1.5.5) and the providers to be used. In this case there is just one:

• Mastercard/restapi — this is the generic REST API interface Provider that we will use to interface to Binary Lane

module "qq_vpc" {
  source = "./modules/vpc"
  vpc_name = "qq_vpc"
}

output "qq_vpc" {
  description = "qq_vpc"
  value = module.qq_vpc.vpc_desc
}

In this section, we are using a Module we will define later in modules/vpc. This module will setup a Virtual Private Cloud in Binary Lane. A VPC is a private subnet. A VPC normally provides security for backend servers by isolating them form the Internet. In our case all the servers we will create will be accessible via the Internet and from the private network. This is not appropriate for a production environment but is ok for a demonstration project such as this.

You can see how we are passing a parameter to the module that names the VPC: vpc_name = “qq_vpc”

In addition, we are taking an output from the Module. In this case it is a description of the VPC we have created and is defined by value = module.qq_vpc.vpc_desc.

module "qq_k8s_master" {
  source = "./modules/public_vps"
  vps_name = "k8s-master"
  vps_flavour = "std-2vcpu"
  vpc_id = module.qq_vpc.vpc_desc.id
}

output "qq_k8s_master" {
  description = "qq_k8s_master"
  value = module.qqk8s_master.v4_ips
}

Now we have our VPC, we can setup our master node VPS within it. This time we are using the public VPS Module with public (ie: Internet facing) networking: modules/public_vps

We are passing the name (k8s-master) and setting the Binary Lane VPS size (std-2vcpu) . We are also passing the VPC that the VPS is to be setup in. To do this we use theid reference from the VPC description that we output earlier (module.qq_vpc.vp_desc.id). Take note of the path name and how it identifies the path as a module (module), the name of the module (qq_vpc), the output from the module (vp_desc) and finally, the field within the output (id).

Like we captured outputs from the VPC creation, we capture the IP address of the created VPS through the output of the Module.

module "qq_k8s_node_1" {
  source = "./modules/public_vps"
  vps_name = "qq_k8s-node-1"
  vps_flavour = "std-1vcpu"
  vpc_id = module.qq_vpc.vpc_desc.id
}

output "qq_k8s_node_1" {
  description = "qq_k8s_node_1"
  value = module.qq_k8s_node_1.v4_ips
}

module "qq_k8s_node_2" {
  source = "./modules/public_vps"
  vps_name = "qq_k8s-node-2"
  vps_flavour = "std-1vcpu"
  vpc_id = module.qq_vpc.vpc_desc.id
}

output "qq_k8s_node_2" {
  description = "qq_k8s_node_2"
  value = module.qq_k8s_node_2.v4_ips
}

In the same way as we created the master node, we now create 2 worker nodes. These use the same Module but you will notice we use a size (flavour) of std-1vcpu this time. This will save costs. You can create as many or as few nodes as you want, each with the size you need.

Terraform has the ability to define multiple resources using array structures but in this project, we have named them separately. We shall see the use of an array in the next section of the file.

resource "local_file" "inventory" {
  filename = "../ansible/inventory"
  content = templatefile("ansible_inventory.tftpl", {
    master_ip = module.qq_k8s_master.v4_ips.public
    node_ips = [
      module.qq_k8s_node_1.v4_ips.public,
      module.qq_k8s_node_2.v4_ips.public
    ]
  })
}

In this final section, we use the ability for Terraform to capture information into a file using a template. In this case the template file is called ansible_inventory.tftpl. Terraform will create a file from this template called inventory, which it will place in the ansible folder.

In this file it will place the public IP addresses of the servers we have created, based on the outputs captured from the Modules. These are required by Ansible in a later article.

REST API Provider

When using the generic REST API Provider (Mastercard/restapi), you need to configure it for the API you are using.

Terraform does not take notice of filenames (only extensions). You can put everything (except Modules) into the same file but for maintenance and readability, we separate them.

Create a file called terraform/rest_api_provider.tf to configure the Provider:

provider "restapi" {
  uri                  = "https://api.binarylane.com.au/v2"
  write_returns_object = true
  debug                = true

  headers = {
    "Authorization" = "Bearer ${var.binarylane_api_key}",
    "Content-Type" = "application/json"
  }

  create_returns_object = true

  id_attribute = "id"

  create_method  = "POST"
  update_method  = ""
  destroy_method = "DELETE"
}

It can be seen that the Provider will use the Binary Lane API. Resource creation returns an object that defines the resource created and has an id field for the unique reference.

The credentials are held in a Terraform global variable binarylane_api_key, which we will define later.

You can also see that Binary lane does not allow updates (update_method = “”) . To update a resource you must delete it and recreate it.

The debug = true line enables additional logging when you run Terraform that can help when using new APIs.

Modules

Earlier we saw that our VPC and VPS resources were created by using Terraform Modules. We now need to define these Modules.

Each Module is created within a subfolder under the modules folder. We will create two:

• vpc — creates a VPC with a private subnet that will contain the VPS nodes that are created
• public_vps — creates a VPS with public and private connections

Each Module consists of three files:

• variables.tf — defines the inputs to the Module (like a method’s parameters)
• main.tf — defines the configuration of the resource to be created in a way that the API accepts
• outputs.tf — defines the outputs from the Module (like a method’s return values)

vpc Module

This Module creates the VPC in which the VPS nodes are created.

You can define the subnet of the VPC through input variables or use the default defined in the variables file.

Create the file: modules/vpc/vpc_variables.tf

This defines the inputs we can set when using the Module in our main.tf file.

variable "vpc_name" {
    description = "Name of the VPC to be created"
    type = string
    default = "my_vpc"
}
variable "vpc_ip_range" {
    description = "Subnet definition for the VPC"
    type = string
    default = "10.240.0.0/16"
}

This Module defines two inputs, vpc_name and the subnet CIDR definition. vpc_ip_range Both are strings and both have default values.

We now need to define what the Module does with these inputs.

Create the file: terraform/modules/vpc/vpc_main.tf

terraform {
    required_providers {
        restapi = { source = "Mastercard/restapi" }
    }
}

resource "restapi_object" "bl_vpc" {
  path = "/vpcs"
  id_attribute = "vpc/id"
  data = <<EOJ
        {
            "name": "${var.vpc_name}",
            "ip_range": "${var.vpc_ip_range}"
        }
    EOJ
}

The Module uses the Mastercard/restapi Provider. The documentation of the provider can be found here.

When it comes to interacting with your infrastructure supplier ( in this case Binary Lane), Terraform uses two terms for its main objects:

• resource — something to be created
• data — information to be retrieved

This Modules creates infrastructure and so defines a resource. The resource is going to be created by the restapi_object within the Provider (you can find this name in the documentation) and we are going to give this instance a name so we can reference it later (bl_vpc).

Having defined our resource, we must now configure it based on the details of the API itself. We set the API endpoint URI and the parameters to be sent (in JSON format). The parameters use the variables passed in to the Module as, for example${var.vpc_name}.

When the Module creates the VPC, the REST API returns a definition of the resource that can be extracted as outputs.

Create the file: modules/vpc/vpc_outputs.tf

This defines the results of the Module once it has called the API endpoint.

output "vpc_desc" {
    description = "The created VPC"
    value = restapi_object.bl_vpc
}

The Module outputs a single value which is the result of creating the VPC. In this case it is an object returned by Binary Lane (restapi_object.bl_vpc) which includes the id of the VPC it created. This id will be used when creating any VPS.

Terraform understands the links between resources and ensures that these dependencies are honoured.

public_vps Module

This Module creates a Virtual Private Server (VPS) within the VPC we just created. In the same way as the VPC Module had three files, so does this one.

Create the file: modules/public_vps/public_vps_variables.tf

variable "vpc_id" {
    description = "ID of the VPC this server is to be created in"
    type = number
    default = 0
}

variable "vps_name" {
    description = "Name of the VPS server to be created"
    type = string
    default = "my_web_svr"
}

variable "vps_flavour" {
    description = "Flavour of the VPS server defining its dimensions"
    type = string
    default = "small"
}

variable "ssh_key" {
    description = "The SSH key to allow access to the VPS"
    type = string
    default = ""
}

This file defines the variables you can use in your Module.

• vpc_id — the id of the VPC to hold this VPS
• vps_name — the name of the VPS
• vps_flavour — the type of VPS (see curl -X GET “https://api.binarylane.com.au/v2/sizes" -H “Authorization: Bearer <your API key>”)

Again, we need to define how a VPS is created.

Create the file: modules/public_vps/public_vps_main.tf

terraform {
    required_providers {
        restapi = { source = "Mastercard/restapi" }
    }
}


resource "restapi_object" "bl_vps" {
  path = "/servers"
  id_attribute = "server/id"
  force_new = ["vps_flavour"]
  data = <<EOJ
        {
            "name": "${var.vps_name}",
            "backups": true,
            "ipv6": false,
            "size": "${var.vps_flavour}",
            "image": "debian-11",
            "region": "syd",
            "vpc_id": "${var.vpc_id}",
            "ssh_keys": ["${var.ssh_key"]
       }
    EOJ
}

This module creates a public VPS within a VPC. The VPS is allocated an internal and public IP address by your clod provider.

Again, the Module uses the Mastercard/restapi provider.

This Module creates a Debian 11 OS server. It also adds your public SSH key to allow you to access it once it is has been created.

Now we define what outputs are created by the Module.

Create the file: modules/public_vps/public_vps_outputs.tf

output "instance_id" {
    description = "ID of the created server"
    value = restapi_object.bl_vps
}

locals {
    v4networks  = jsondecode(restapi_object.bl_vps.api_response).server.networks.v4
    v4_private_ip = local.v4networks[index(local.v4networks.*.type, "private")].ip_address
    v4_public_ip = local.v4networks[index(local.v4networks.*.type, "public")].ip_address
}

output "v4_ips" {
    description = "IP V4 addresses of VPS"
    value = {
        public = local.v4_public_ip,
        private = local.v4_private_ip
    }
}

This sets values that can be used by Terraform elsewhere using a ‘path’ as follows:

module.<name of instance>.<output name>.<field name>

eg: module.qq_k8s_node_2.v4_ips.public

Like Java methods, local variables can be created using the locals block. Variables are defined and referenced in the Go language format. This means that the JSON returned by a provider must be decoded using the built in jsondecode function.

We now have:

• The main.tf file that defines what we want (1 x VPC, 3 x VPS)
• The vpc Module that creates a VPC
• The public_vps Module that creates a VPS with internal and external IP addresses

Now we need to define variables that allow us to pass values into the Terraform configuration that we may not want to commit to our repo.

Variables

When you create Terraform configurations, you may want to pass in variables, such as credentials. You do not want to commit these to your code repos.

Terraform allows you to define variables (with types and default values) and then to set these variables in a separate file.

Defining the Variables

Create the file variables.tf:

variable "binarylane_email" {
  description = "Email Address/User used to login to the BinaryLane api."
  default = "default email"
}

variable "binarylane_api_key" {
  description = "API key used to login/verify for APIs used at binarylane."
  default = "default key"
}

variable "ssh_key" {
  description = "The PUBLIC SSH key you want to use to access your VPSs."
  default = "default key"
}

Setting the Variables

Now you have defined the variables, you can now set them.

Create the file terraform.tfvars:

binarylane_email = "<your Binary Lane account email address"
binarylane_api_key = "<your Binary Lane API key>"
ssh_key = "<the SSH key you created>"

You can see that this should not be in your git repo due to the credentials it holds.

Binary Lane API Key

This is the key you created ealier.

SSH Key

It is advisable that you connect to each VPS using a separate SSH key to your normal key. For this reason you should create a key now.

cd ~/.ssh
ssh-keygen -b 4096 -t rsa

Enter a filename of qq_rsa. Do not give it a passphrase.

You can now use ~/.ssh/qq_rsa.pub as the public SSH key. Log in to your Binary Lane account and go to:

https://home.binarylane.com.au/ssh-keys

Click + Create Key and enter a name (eg: Quick Queue). Copy your public key contents into the box and click Create.

When you have done this, it will be shown with a fingerprint. Copy the fingerprint into the terraform.tfvars file.

Remember to keep the qq_rsa file secure and private.

Additional Files

There are two additional files that are required.

terraform/ansible_inventory.tftpl

This is the template file that Terraform will fill out to create an inventory file for ansible based on the results of the VPS creation.

[k8s_master]
${master_ip}

[k8s_node]
%{ for addr in node_ips ~}
${addr}
%{ endfor ~}

You can see that Terraform can substitute single values (eg: ${master_ip}) as well as arrays of values (${addr}).

.gitignore

This is the standard .gitignore, ensuring files are not committed to git.

# Local .terraform directories
**/.terraform/*
.terraform.*.hcl

# Local keys directories
**/keys/*

# .tfstate files
*.tfstate
*.tfstate.*

# Crash log files
crash.log
crash.*.log

# Exclude all .tfvars files, which are likely to contain sensitive data, such as
# password, private keys, and other secrets. These should not be part of version 
# control as they are data points which are potentially sensitive and subject 
# to change depending on the environment.
*.tfvars
*.tfvars.json

# Ignore override files as they are usually used to override resources locally and so
# are not checked in
override.tf
override.tf.json
*_override.tf
*_override.tf.json

# Include override files you do wish to add to version control using negated pattern
# !example_override.tf

# Include tfplan files to ignore the plan output of command: terraform plan -out=tfplan
# example: *tfplan*

# Ignore CLI configuration files
.terraformrc
terraform.rc

The important thing to note here is that the .tfstate files are not committed.

It is important to keep hold of this file as it tells Terraform which resources have been created. Based on this Terraform can decide what to do.

It should not be committed to your git repository as it may contain sensitive information.

You will also notice that the .terraform folder is not committed. This holds the provider and Module information. Each time you create or modify a Provider or Module, Terraform will update the values here as described in the next section.

Running Terraform

Check that you now have the following files and folder in the terraform folder:

terraform
  modules
    public_vps
      public_vps_main.tf
      public_vps.outputs.tf
      public_vps.variables.tf
    vpc
      vpc_main.tf
      vpc.outputs.tf
      vpc.variables.tf
  .gitignore
  ansible_inventory.tftpl
  main.tf
  rest_api_provider.tf
  terraform.tfvars
  variables.tf

Now you have defined your infrastructure as code, you may consider pushing it to your IaC repository.

You can find my version here:

https://github.com/MartinHodges/Quick-Queue-IaC/tree/part3

Running Terraform is a three step process:

1. init
2. plan
3. apply

Step 1: init

This step tells terraform to download the provider plugins it needs for the main configuration and any Modules. It also installs those Modules so they can be used.

Whenever. you add a provider or a Module, you will need to run the init step again.

You can run the init step at anytime without harming your current infrastructure.

It is now time to initialise Terraform from the command line in your terraform folder. Whilst the init step will also validate your configuration, you can check it first with:

terraform validate

The errors can sometimes seem obscure but if you read the whole error report, you will find that it is quite accurate.

You can ignore errors referring to missing Providers as we have not yet initialised the .terrraform folder. We can do that now with:

terraform init

If everything ran successfully, you should see a message saying:

Terraform has been successfully initialized!

You will start to find more hidden subfolders in your terraform folder.

Step 2: plan

This step asks Terraform to show you what it is going to do, without doing anything. It is like a dry run. Not all information is available as some of it is only available once the resources have been created (eg: IP addresses) but you can see what it is planning on doing.

Plan now with:

terraform plan

There is a lot of output from this comand but you should see that it is saying it will create:

• local_file.inventory
• module.qq_k8s_master.restapi_object.bl_vps
• module.qq_k8s_node_1.restapi_object.bl_vps
• module.qq_k8s_node_2.restapi_object.bl_vps
• module.qq_vpc.restapi_object.bl_vpc

It will also tell you:

Plan: 5 to add, 0 to change, 0 to destroy.

Below this will be a series of outputs. Remember we put them in the main.tf file? When you place outputs at the upper most level, Terraform will print them out to the console. Most of it will be marked as (known after apply) as it needs to get them from Binary Lane.

So far we have not created anything at all.

Dependencies

Some resources (like our VPS) will be dependent on other resources (like our VPC). Terraform automatically detects these dependencies and creates the resources in the order needed.

There are situations where you can give Terraform dependencies but it is best not to do this as Terraform is declarative and not imperative.

Step 3. apply

The final step is to apply the configuration you have asked for.

Note that this may not do what the plan step said it would do, as Terraform always replans before applying your configuration. This ensures any change made to the configuration will be picked up. For this reason, planning is optional but recommended.

Apply your configuration (Note that this will start incurring costs with your cloud supplier, Binary Lane in this case).

terraform apply

After typing this command, it will show you the latest plan. It will then ask you to confirm that you want it to create the infrastructure in accordance with this plan.

If you want to, type yes and press return.

All going well, you will then see:

Apply complete! Resources: 4 added, 0 changed, 0 destroyed.

The outputs in the main.tf file will then show the details of what was created. You can also check your Binary Lane account and see the VPC with and 3 VPCs created.

You should also have a file in the ansible folder called inventory with the IP addresses of each of the VPS nodes in it.

You are now ready to configure the servers to install Kubernetes. Before you start the configuration, it is important that the new servers are added to your known_hosts file. The easiest way to do this and to prove your servers have been set up correctly is to ssh into each of them using:

ssh -i ~/.ssh/qq_rsa root@<ip address>

Answer yes when it asks to confirm the finger print of each. You should be able to log in without a password. If you have any problems, you can go to the Binary Lane services page and reset the password. You can then use that to log in to the web console and try and work out what is wrong.

Step 4. destroy

Ok, so I said there were only 3 steps. There is a 4th — destroy. This step deletes all the resources created previously.

This is an unrecoverable step and all your configuration and data will be lost.

To delete your infrastructure, you can enter:

terraform destroy

Like the apply step, it will tell you what it is going to delete and will ask you to enter yes. When you do, your infrastructure will be deleted.

You can then do step 3 to recreate the system again.

Summary

In this article, we created a set of configuration files that Terraform can use to create the infrastructure required for a Kubernetes, 3 node cluster.

Using a generic REST Provider, Terraform called the Binary Lane API to create the VPC and VPS nodes that we requested.

Finally we saw how the infrastructure can be torn down with the destroy command.

In the next article, we look at updating the OS on each node and then deploying a Kubernetes cluster on it.

Series Introduction

Previous — Project Setup

Next — Configuring Servers using Ansible