Terraform Pipeline with Gitlab CI and OpenID Connect

As a developer or a DevOps Engineer, if you are working with software deployed in the cloud, you must probably be in a scenario where you need to find a way to configure your deployment pipelines and have a way to allow your CI system to authenticate to the cloud from your pipeline.

Let’s say we are working with AWS cloud and want to configure our pipeline to authenticate with AWS and provision some resources via Terraform. We have different ways we can do this:

1. We can run our CI workers on AWS with an instance profile. But we don't want to give the same access to each pipeline or build on every branch of that pipeline. For this, we would need to run multiple workers that assume different roles and also limit various jobs to run on the correct workers.
2. Configure our CI system with access keys for an AWS IAM user. This is commonly used I must say. With this, we allow our CI system to use those credentials to run our pipelines. One big challenge is these credentials don’t expire by default, so we need to set up a credentials rotation process that will ensure we don't have years-old credentials hanging around but we do this in theory 😂.
3. Use OIDC Identity Provider to authenticate using dynamic, short-lived credentials — the focus of this demo

What is OpenID Connect

OpenID Connect (OIDC) is a standard authentication mechanism based on the OAuth 2.0 protocol. It’s used across the internet for user authentication e.g Sign in with Google, Sign in with Instagram or Sign in with Facebook.

Prerequisites

You need to have some basic knowledge of working with Terraform and Gitlab CI.

• Terraform installed in your local machine
• A Gitlab Repository
• A working AWS Account. You can signup for a free tier
• And of course a code editor of your choice.

Introduction

Today we are going to demonstrate OIDC authentication with the following:

• Gitlab CI for the pipeline but you can substitute these with your favourite CI tool (Azure Pipelines or CircleCI)that acts as an identity provider(IdP)
• Amazon Web Services as the authentication target. The same applies to any cloud provider of your choice e.g Google Cloud Platform, Azure.
• Terraform to configure everything

Gitlab with AWS OIDC

Step 1: Add Gitlab as an OIDC provider

The first thing we need to do is configure AWS with the OIDC provider so we can later use it with IAM roles. Please pay attention to the variables that we will add later if you need to configure them for a different CI tool.

Gitlab tls certificate

oidc config

Step 2: Create an IAM role

The next thing needed is to allow certain builds to assume an IAM role. We do this by creating an IAM and specifying a trust policy. The “Assume Role Policy” specify who or what can assume the role.

Our assume role policy says that builds running on the main branch of the calvine-devops/aws/oidc-test-project project on GitLab.com can assume this role.

IAM Role

Now that we have the role, we then attach a policy to it. Our policy is allowing anyone (or anything) that assumes the role to do what it needs to do. In our case, this policy allows some basic S3 operation on a particular bucket oidc-test-bucket

IAM policy

Variables File

Let’s add some of the variables we have referenced in our configs.

Variables

Outputs

I always use the remote backend for storing state files. You can remove that if that is not what you want to use.

Versions

Provider

Create a backend file backend.tfvarsand initialize terraform via

terraform init -backend-config=backend.tfvars

Run Plan

terraform plan -out gitlab-oidctfplan 

Run Apply

terraform apply gitlab-oidctfplan

Successful apply

To Destroy

terraform destroy -auto-approve 

Step 3: Assume the role vis Gitlab CI

We need two standard environment variables that are used in nearly every AWS client configuration.

• AWS_ROLE_ARN — Defines the role to be assumed
• AWS_WEB_IDENTITY_TOKEN_FILE — Defines a file that will be used to read the token.

Gitlab CI config

Grab your AWS_ROLE_ARN from Terraform output and add it to the variables section.

Successful Pipeline when we list the content of the S3 bucket we granted access.

Successfully CI pipeline

Failed Pipeline when we try to list the content of the S3 bucket we didn’t grant access.

Failed CI pipeline

This is all for now. I hope you have learnt something and enjoyed reading the article. Till next time.

Here is the repo for this article. Follow me on GitHub for more about DevOps and DevSecOps and GitOps.

Thanks for reading. Let’s connect on Twitter and LinkedIn 😁.