Automating multi-environment deployments using Terraform, Gitlab Pipelines, and Git Flow

Automation describes a wide range of technologies that reduce human intervention in processes. Human intervention is reduced by predetermining decision criteria, subprocess relationships, and related actions — and embodying those predeterminations in machines.

Managing multiple environments was a challenging task a few years ago as it required constant updates on documentation and well-defined processes in order to achieve a sustainable and reliable system.

Thanks to the recently created DevOps approach, there are a set of tools and technologies for various aspects of the software development lifecycle (SDLC). Now we can automate many repetitive tasks and leave them to machines, which are good at doing that and ultimately designed for this purpose.

Modern software engineering requirements nowadays are focused on delivering business value at a faster pace than before as Agile methodologies are now a standard way of developing and delivering software across many companies.

With this in mind, some tools and processes became a standard stack across new software projects, especially cloud-native systems. Some of those tools and processes are Terraform, Gitlab Pipelines, and Git Flow. Let’s take a closer look into each of those:

Terraform

Terraform is an open-source infrastructure as code software tool that provides a consistent CLI workflow to manage hundreds of cloud services. Terraform codifies cloud APIs into declarative configuration files.

A Terraform configuration is a complete document in the Terraform language that tells Terraform how to manage a given collection of infrastructure. A configuration can consist of multiple files and directories. Here is an example:

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 1.0.4"
    }
  }
}

variable "aws_region" {}

variable "base_cidr_block" {
  description = "A /16 CIDR range definition, such as 10.1.0.0/16, that the VPC will use"
  default = "10.1.0.0/16"
}

variable "availability_zones" {
  description = "A list of availability zones in which to create subnets"
  type = list(string)
}

provider "aws" {
  region = var.aws_region
}

resource "aws_vpc" "main" {
  # Referencing the base_cidr_block variable allows the network address
  # to be changed without modifying the configuration.
  cidr_block = var.base_cidr_block
}

The example above describes a simple network topology for Amazon Web Services, just to give a sense of the overall structure and syntax of the Terraform language. Similar configurations can be created for other virtual network services, using resource types defined by other providers, and a practical network configuration will often contain additional elements not shown here.

A very interesting and useful configuration option available in Terraform is called workspaces. Using this option, we will be able to manage multiple environments easily and without conflicts between different versions of the same system.

Each Terraform configuration has an associated backend that defines how operations are executed and where persistent data such as the Terraform state are stored.

The persistent data stored in the backend belongs to a workspace. Initially, the backend has only one workspace, called “default”, and thus there is only one Terraform state associated with that configuration.

Certain backends support multiple named workspaces, allowing multiple states to be associated with a single configuration. The configuration still has only one backend, but multiple distinct instances of that configuration can be deployed without configuring a new backend or changing authentication credentials.

Gitlab Pipelines

GitLab CI/CD is a tool for software development using the continuous methodologies:

• Continuous Integration (CI)
• Continuous Delivery (CD)
• Continuous Deployment (CD)

By using a GitLab CI/CD Pipeline, you can catch bugs and errors early in the development cycle. Ensure that all the code deployed to production complies with the code standards you established for your app.

To use GitLab CI/CD:

1. Ensure you have runners available to run your jobs. If you don’t have a runner, install GitLab Runner and register a runner for your instance, project, or group.
2. Create a .gitlab-ci.yml file at the root of your repository. This file is where you define your CI/CD jobs.

When you commit the file to your repository, the runner runs your jobs. The job results are displayed in a pipeline.

Here is an example of a .gitlab-ci.yml file:

build-job: 
 stage: build 
 script: 
  — echo “Hello, $GITLAB_USER_LOGIN! Les't build your system now.” 
  - make package
test-job:  
 stage: test 
 script: 
  — echo “Now, let's run some unit tests.”
  - make test 
plan-job: 
 stage: plan 
 script: 
  — echo “Let's make plan about what we will published.”
  - terraform plan
deploy-job: 
 stage: deploy 
 script: 
  — echo “Deploying latest changesfrom the $CI_COMMIT_BRANCH.”
  - terraform apply

In simple terms, a Gitlab Pipeline is responsible for “getting hands dirty”, in the context of executing repetitive work and providing results in a structured way.

When we are using a multi-environment deployment, we might have different tasks for each environment and this can be done by setting new jobs and stages in the .gitlab-ci.yml file.

For detailed information on how to set up a Gitlab Pipeline, refer to Gitlab Pipeline Quick Start page.

Git Flow

Git Flow is an alternative Git branching model that involves the use of feature branches and multiple primary branches. It was first published and made popular by Vincent Driessen at nvie.

Compared to trunk-based development, Git Flow has numerous, longer-lived branches and larger commits. Under this model, developers create a feature branch and delay merging it to the main trunk branch until the feature is complete. These long-lived feature branches require more collaboration to merge and have a higher risk of deviating from the trunk branch. They can also introduce conflicting updates.

By using Git Flow branching model, we are able to isolate different versions of the same system and have them available for deployment on your cloud provider, without having the stress of managing different versions using multiple folders or repositories for that purpose.

Using this process combined with Terraform Workspaces and Gitlab Pipelines, we can design a highly automated and reliable deployment strategy to keep development, testing and production environments always up-to-date with the latest changes in a few minutes.

In my next article about this topic, will go step-by-step over an example configuration using all these services and processes so you can adapt to your team and start using this model on your projects.

Follow me here and on Twitter to stay tuned for the latest updates.