Terraform module development workflow

Organisations have embraced the idea of infrastructure as code and it has undoubtedly become fundamental in a world where businesses have to deliver high-quality software fast in order to remain competitive.

One of the biggest benefits of delivering our infrastructure programatically is to enable teams to reliably construct any version of their infrastructure from their versioned code at any time. It is also a considerable advantage being able to abstract common blocks of configuration into reusable infrastructure code.

While treating our infrastructure as code provides many benefits, it is also worth mentioning it has its own downside. Code sprawl and complexity creates its own maintenance and quality challenges.

But don’t let these challenges draw you back. In this article we are going to focus on the practices and concepts already adopted by code development to deliver software reliably and repeatedly. We will take a detailed look on how continuous integration can help us establish a solid delivery pipeline for our terraform modules.

The workflow

The diagram below shows the desired workflow we want to achieve

We will be assuming that our module code is located in a repository under GitHub.com and the CI Tool that will be orchestrating the delivery pipeline is Jenkins.

The branching strategy

We will be using the gitflow branching strategy to handle changes in our repository. This involves the usage a Master branch that will be protected, a develop branch from where we will create features branches to push new changes. The image below shows 2 changes currently being done on our repository through the usage of two different feature branches:

When a new change needs to be made to our module a new branch will be created from develop with the format feature/*. Once we are happy with the changes, a Pull Request (PR) will be created against the develop branch to merge the changes back. Github will be configured to trigger a web-hook with Jenkins every time a PR is created or modified.

Once Jenkins is triggered it will look for a Jenkinsfile inside the repository and create a pipeline based on the configuration. Once the pipeline is available a new build will be triggered.

While the pipeline is being executed the PR in github will be locked, waiting for an outcome before it allows us to merge our feature/*branch into develop. If the build is successful the merge will go green and allow the changes to be merged, otherwise it will go red and not allow show that there was an error. This way we ensure that any changes made to our modules will only go back to the repository only if all validations are passed successfully.

Setting up the repository

We are taking into consideration that we already have a github repository with our module files inside. In order to have Jenkins building a pipeline every time we push changes into our code we need to create a Jenkinsfile inside of it with the following code:

pipeline {
    agent any 
    stages {
        stage('Terraform Init') {
            steps {
                sh 'terraform init -input=false'
            }
        }
        stage('Validations'){
            parallel {
                stage('Validate Terraform configurations') {
                    steps {
                        sh 'find . -type f -name "*.tf" -exec dirname {} \\;|sort -u | while read m; do (terraform validate -check-variables=false "$m" && echo "√ $m") || exit 1 ; done'
                    }
                }
                stage('Check if Terraform configurations are properly formatted') {
                    steps {
                        sh "if [[ -n \"\$(terraform fmt -write=false)\" ]]; then echo \"Some terraform files need be formatted, run 'terraform fmt' to fix\"; exit 1; fi"
                    }
                }
                stage('Check Terraform configurations with tflint'){
                    steps {
                        // To install tflint
                        // curl -L -o /tmp/tflint.zip https://github.com/wata727/tflint/releases/download/v0.4.2/tflint_linux_amd64.zip && unzip /tmp/tflint.zip -d /usr/local/bin
                        sh  "tflint"
                    }
                }
            }
        }
        stage('Terraform Kitchen') {
            steps {
                sh 'kitchen test'
            }
        }
    }
}

Setting up Terraform Kitchen

Testing Terraform Modules using Kitchen Terraform is useful for assuring the created reusable terraform scripts are working as expected. Once we’ve added our desired tests inside of our repository alongside the Jenkinsfile they should be picked up automatically on the last stage of the pipeline.

Setting up the web-hook

In order to configure Jenkins to be triggered every time a change is pushed into our repository, we need to set up a week-hook. This can be done under in github under Settings → Hook & services → Add webhook

Where Payload URL is our root Jenkins URL followed by /github-webhook/

Also we need to be sure to only let Pull Requests to trigger the web-hook as shown below

We need to configure our Jenkins master to also manage web-hooks from the general configuration

Blocking pull requests

We also need to configure our repository to block Pull Requests to be merged if our pipeline does not execute successfully. In order to achieve this we need to navigate to our repository’s Settings → Branches and click on Require status checks to pass before merging under our develop branch, we also need to make sure that continuous-integration/jenkins/pr-head is selected

The image below shows the correct configuration to achieve this:

The workflow in action

When a new PR (pull request) is created, Github will automatically block the Merge pull request option and wait until all the checks have been completed, in this case the Jenkins pipeline.

The Jenkinsfile pipeline created will look as follows under Jenkins when all stages are executed successfully

Once the pipeline has been successfully executed we can see under our PR that the Merge pull requests option has been enabled

If an error occurs, the individual stage will be marked with a cross. By clicking on it we can see the logs. The rest of the pipeline will be skipped.

When the pipeline fails the Pull Request will be blocked in github.

Conclusion

The process shown above allows for rapid management of change, early testing (fast failure detection) and continuous refinement of the code. It also enables collaboration between teammates while generating stable reusable blocks of code that everyone can take advantage of.