Open in app

Sign in

Write

Sign in

Encryption of terraform state using OpenTofu

Navratan Lal Gupta
Linux Shots
Published in
11 min readJun 3, 2024

Opentofu is an opensource fork of terraform maintained by SpaceLift under project of Linux Foundation.

OpenTofu 1.7 brings a long pending feature in terraform which had been requested by users for very long time. This feature allows client-side encryption of terraform state.

In current terraform state, State file including sensitive values in states are stored in plain text in terraform backend storage. This feature of OpenTofu adds a layer of security by encrypting terraform state using AES GCM v2 encryption method at client side itself.

Client-side encryption with OpenTofu

Contents

  1. Pre-requisites
  2. Why it works on my machine?
  3. Install Opentofu
  4. Demo — Encrypt new tfstate
  5. Demo — Change encryption key for encrypted terraform state
  6. Demo — Remove terraform state encryption
  7. Demo — Encrypt an existing plaintext terraform state

Pre-requisites

  1. Knowledge of terraform
  2. Knowledge of any of cloud providers (AWS, GCP, Azure, etc)
  3. You must have cloud credentials setup on your system

Why it works on my machine?

I am using:

  1. Ubuntu 24.04 OS on my system
  2. OpenTofu version 1.7.1
  3. AWS terraform module (hashicorp/aws) v5.52.0
  4. Minio (S3-API compatible object storage) for storing terraform state
  5. AWS cloud
System information

Install OpenTofu

To install OpenTofu on your system, Follow link https://opentofu.org/docs/intro/install/ and follow the guide as per your Operating system.

I am using DEB package to install OpenTofu in my Ubuntu system. To install this on Ubuntu OS, Run below commands

# Download installation script
curl --proto '=https' --tlsv1.2 -fsSL https://get.opentofu.org/install-opentofu.sh -o install-opentofu.sh

# Give execute permission to script
chmod +x install-opentofu.sh

# You can check content of script
cat install-opentofu.sh

# Install Opentofu
./install-opentofu.sh --install-method deb
Install Opentofu

Demo — Encrypt new terraform state

For this demo purpose, I will create an AWS Dynamo DB table (Its free upto 25GB of storage) using OpenTofu.

Lets start with writing a simple terraform provider block first. This block will have cloud providers and terraform state backend configuration. Basic terraform provider block for AWS will look like below. You can create a providers.tf file for provider block.

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.0"
    }
  }

  backend "s3" {
    bucket = "tfstate-bucket-linuxshots" # S3 bucket in which terraform state will be stored
    key = "dynamodb/terraform.tfstate"
    region = "ap-south-1"
  }
}

# Configure the AWS Provider
provider "aws" {
  region = "ap-south-1"
}

Now, To enable encryption of terraform state, We need to add encryption block in same providers.tf file.

encryption block will resides under terraform block.

Encryption block consists of 4 parts:

  1. key_provider: This part of encryption block have the configuration related to key providers which will hold keys to encrypt and decrypt the state. Key provider can be PBKDF2 (passphrase), AWS KMS, GCP KMS or OpenBao. Key provider can also be defined using variable TF_ENCRYPTION.
  2. method: This part of encryption block defines the encryption method. Currently OpenTofu only supports AES-GCM encryption method. By default, It uses AES-256 bit encryption. There is one more method called unencrypted which is used to migrate a plain text state to encrypted state or vice-versa. We will talk about this in further sections.
  3. state: This part of encryption block defines which method to use for encrypting state and whether or not to encrypt state file.
  4. plan: This part of encryption block defines which method to use for encrypting plan file and whether or not to encrypt plan file.

In this demo, I will be using a passphrase (PBKDF2 key provider) to encrypt the state file.

After adding encryption block in terraform block, our providers.tf file looks like this. Comments added in below code will help understand better.

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.0"
    }
  }

  backend "s3" {
    bucket = "tfstate-bucket-linuxshots"
    key = "dynamodb/terraform.tfstate"
    region = "ap-south-1"
  }

  # Encryption block
  encryption {

    # Key provider
    # Encryption key can also be provided using variable. Like,
    # export TF_ENCRYPTION='key_provider "pbkdf2" "state_encryption_password" { passphrase = "Th!s1sMyp@$sw0Rd" }'
    # When using variable TF_ENCRYPTION, Comment the key_provider block from here
    key_provider "pbkdf2" "state_encryption_password" {
        passphrase = "Th!s1sMyp@$sw0Rd"
    }

    # Encryption method. Currently only aes_gcm mode of encrypttion is supported
    method "aes_gcm" "encryption_method" {
        keys = key_provider.pbkdf2.state_encryption_password
    }


    # Encrypt state file or not
    state {
        # If enforced is true, state will be encrypted
        enforced = true
        method = method.aes_gcm.encryption_method
    }

    # Encrypt plan file or not
    plan {
        # If enforced is true, plan will be encrypted
        enforced = false
    }
  }
}

# Configure the AWS Provider
provider "aws" {
  region = "ap-south-1"
}

Lets add code for Dynamo DB resource in terraform to create a Dynamo DB table. I will be writing terraform code Dynamo DB in dynamodb.tf file.

resource "aws_dynamodb_table" "myddtable" {
  name = "my_data"
  hash_key = "my_data_hash_key"
  attribute {
    name = "my_data_hash_key"
    type = "S"
  }
  read_capacity = 20
  write_capacity = 20
}

Let’s initialize the terraform/OpenTofu now.

tofu init
Initialize OpenTofu

Once initialization is completed, Lets create dynamo db table using OpenTofu.

tofu apply

# Type "yes" if plan looks good
Create Dynamo DB using Opentofu (tofu)

Once tofu apply is successfully completed. Let’s check if Dynamo DB is created in AWS.

Dynamo DB created in AWS

Lets see how our state file in Minio (S3-API compatible bucket) looks like.

Terraform state file in bucket

If we check content of terraform state file, The encrypted content would look something like below.

Encrypted State file

We can also use variable TF_ENCRYPTION instead of using key_provider block in code. If you plan to use this variable, make sure to remove key_provider block from code and run the commands as below:

# If we do not want to use key_provider block in code
# and instead use variable to define key_provider and passphrase.

export TF_ENCRYPTION='key_provider "pbkdf2" "state_encryption_password" { passphrase = "Th!s1sMyp@$sw0Rd" }'
tofu init
tofu apply

Demo — Change encryption key for encrypted terraform state

We may sometime need to rotate the key. This will require re-encryption of state file with new key.

To change encryption key (key provider) of a encrypted terraform state file, we need to use fallback block inside state block.

Do the following in encryption block of providers.tf.

First, Add one more key_provider with new passphrase or KMS.

Then, Add one more method which uses new key_provider added in previous step.

Then, Under state block, add a fallback block which will be pointing to existing method and replace the method defined in state block with new method.

Below code with comments will help you understand better. This is how our providers.tf block will looks like. Try to understand content of encryption block.

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.0"
    }
  }

  backend "s3" {
    bucket = "tfstate-bucket-linuxshots"
    key = "dynamodb/terraform.tfstate"
    region = "ap-south-1"
  }

  # Encryption block
  encryption {

    # Current Key provider
    key_provider "pbkdf2" "state_encryption_password" {
        passphrase = "Th!s1sMyp@$sw0Rd"
    }

    # New Key provider
    key_provider "pbkdf2" "state_encryption_new_password" {
        passphrase = "Th!s1sMyp@$sw0Rd"
    }

    # Current Encryption method
    method "aes_gcm" "encryption_method" {
        keys = key_provider.pbkdf2.state_encryption_password
    }

    # New Encryption method
    method "aes_gcm" "new_encryption_method" {
        keys = key_provider.pbkdf2.state_encryption_password
    }

    # Encrypt state file or not
    state {
        enforced = true
        method = method.aes_gcm.new_encryption_method  # New encryption method

        # fallback block
        fallback {
          method = method.aes_gcm.encryption_method    # Current encryption method in fallback block
        }
    }

    plan {
        enforced = false
    }
  }
}

# Configure the AWS Provider
provider "aws" {
  region = "ap-south-1"
}

After updating providers.tf file. Run below command to re-encrypt the state file with new key. Running refresh command may take sometime due to re-encryption process.

tofu init
tofu refresh

After terraform state is re-encrypted, We can now remove the old key_provider block, old method block from encryption block and fallback block from state block inside providers.tf file.

After removing above contents from providers.tf file, File content will look something like this.

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.0"
    }
  }

  backend "s3" {
    bucket = "tfstate-bucket-linuxshots"
    key = "dynamodb/terraform.tfstate"
    region = "ap-south-1"
  }

  # Encryption block
  encryption {

    # New Key provider
    key_provider "pbkdf2" "state_encryption_new_password" {
        passphrase = "Th!s1sMyp@$sw0Rd"
    }

    # New Encryption method
    method "aes_gcm" "encryption_method" {
        keys = key_provider.pbkdf2.state_encryption_password
    }

    # Encrypt state file or not
    state {
        enforced = true
        method = method.aes_gcm.new_encryption_method  # New encryption method
    }

    plan {
        enforced = false
    }
  }
}

# Configure the AWS Provider
provider "aws" {
  region = "ap-south-1"
}

You can run tofu init and tofu plan to check if re-encrypted state file is working fine.

Demo — Remove terraform state encryption

We may sometime need to decrypt the terraform state. Though it could be security threat to keep unencrypted state file in backend.

Here is how we can decrypt and store the terraform state file in plaintext from an already encrypted terraform state file in backend.

There is one more type of method available other than aes_gcm. It is named unencrypted. This method is used to migrate an encrypted state file to plain text state file, Or, a plain text state file to encrypted state file.

To remove encryption from a state file, do the following in encryption block of providers.tf file.

First, Add one more method block of type unencrypted and keep this block empty. This block does not have any attributes.

Then, Under state block, add a fallback block which will be pointing to existing method with current passphrase. Replace the method defined in state block with new unencrypted method. Also, remove enforced = true from state block.

Below code with comments will help you understand better. This is how our providers.tf block looks like. Try to understand the content of encryption block.

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.0"
    }
  }

  backend "s3" {
    bucket = "tfstate-bucket-linuxshots"
    key = "dynamodb/terraform.tfstate"
    region = "ap-south-1"
  }

  # Encryption block
  encryption {

    # Current Key provider
    key_provider "pbkdf2" "state_encryption_password" {
        passphrase = "Th!s1sMyp@$sw0Rd"
    }

    # Current Encryption method
    method "aes_gcm" "encryption_method" {
        keys = key_provider.pbkdf2.state_encryption_password
    }

    # New method block of type unencrypted. This will be empty block.
    method "unencrypted" "plaintext" {}

    state {
        method = method.unencrypted.plaintext  # New unencrypted method

        # fallback block
        fallback {
          method = method.aes_gcm.encryption_method    # Current encryption method in fallback block
        }
    }

    plan {
        enforced = false
    }
  }
}

# Configure the AWS Provider
provider "aws" {
  region = "ap-south-1"
}

After updating providers.tf file, Run below commands to decrypt the terraform state file in backend.

tofu init
tofu refresh

You can get to backend bucket and check the content of terraform state file. You should be able to see terraform states in plain text like below.

After state file is encrypted, Remove entire encryption block from providers.tf file and re-run tf init, tofu refresh and tofu plan to validate.

After removing encryption block, providers.tf file will look something like below.

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.0"
    }
  }

  backend "s3" {
    bucket = "tfstate-bucket-linuxshots"
    key = "dynamodb/terraform.tfstate"
    region = "ap-south-1"
  }
}

# Configure the AWS Provider
provider "aws" {
  region = "ap-south-1"
}

Demo — Encrypt an existing plaintext terraform state

Client-side encryption of terraform state is very new feature introduced in OpenTofu. So there must be many existing projects which would be using terraform state file in plaintext and is stored in backend storage bucket.

To migrate those plaintext state files to encrypted terraform state, We need:

  1. If terraform state has been created by terraform and not OpenTofu then, We will first need to migrate our terraform code and state to OpenTofu code and state. Codes for both are same except few changes. You can follow this document to migrate to OpenTofu — https://opentofu.org/docs/intro/migration/
  2. To encrypt an existing state file, We will need to have some change applied on resource. So there must be some change which can be applied in any of resources in terraform state.
  3. If our state file is already created by OpenTofu or has already been migrated to OpenTofu, Then follow below steps

First, Add encryption block under terraform block.

Then, Add key_provider block inside encryption block. This block defines which key provider are we going to use to encrypt the state. Key providers can be any of pbkdf2 , aws_kms, gcp_kms or openbao. Check the document — https://opentofu.org/docs/language/state/encryption/#key-providers for details.

Then, Add method block of aes_gcm type under encryption block. Document — https://opentofu.org/docs/language/state/encryption/#aes-gcm

Then, Add one more empty method block of type unencrypted under encryption block.

Then, Add a state block inside encryption block with method value as aes_gcm method. Also, Add a fallback block inside state block and add method value as unencrypted method.

Below is an example of my code.

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.0"
    }
  }

  backend "s3" {
    bucket = "tfstate-bucket-linuxshots"
    key = "dynamodb/terraform.tfstate"
    region = "ap-south-1"
  }

  # Encryption block
  encryption {
    
    # Key provider to use for encryption
    key_provider "pbkdf2" "state_encryption_password" {
        passphrase = "Th!s1sMyp@$sw0Rd"
    }

    # method block to define encryption method
    method "aes_gcm" "encryption_method" {
        keys = key_provider.pbkdf2.state_encryption_password
    }

    # Empty method of type unencrypted
    method "unencrypted" "plaintext" {}
    
    # State block
    state {
        method = method.aes_gcm.new_encryption_method  # Encryption method to use for encryption

        # Fallback block
        fallback {
          method = method.unencrypted.plaintext # Fallback block pointing to empty unencrypted method
        }
    }
  }
}

# Configure the AWS Provider
provider "aws" {
  region = "ap-south-1"
}

Now I will update the configuration of dynamo db and run tofu init and tofu apply to encrypt the terraform state.

Once, It is done, remove empty method block of unencrypted type from encryption block, and remove fallback block from state bock. And then, add enforced = true in state block.

Our final terraform block will look something like below:

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.0"
    }
  }

  backend "s3" {
    bucket = "tfstate-bucket-linuxshots"
    key = "dynamodb/terraform.tfstate"
    region = "ap-south-1"
  }

  # Encryption block
  encryption {
    
    # Key provider to use for encryption
    key_provider "pbkdf2" "state_encryption_password" {
        passphrase = "Th!s1sMyp@$sw0Rd"
    }

    # method block to define encryption method
    method "aes_gcm" "encryption_method" {
        keys = key_provider.pbkdf2.state_encryption_password
    }
    
    # State block
    state {
        enforced = true
        method = method.aes_gcm.new_encryption_method  # Encryption method to use for encryption
    }
  }
}

# Configure the AWS Provider
provider "aws" {
  region = "ap-south-1"
}

Then, Re-run tofu init and tofu apply. Now we have migrated to encrypted terraform state from plaintext terraform state.

I hope this article must have helped you learn something new and improve your IaaC security. You can find complete documentation on state encryption on OpenTofu’s official doc — https://opentofu.org/docs/language/state/encryption/

Thank you for reading until the end. Before you go:

  1. Please consider clapping and following me(author)! 👏
  2. Subscribe to email notifications for new articles: https://navratangupta.medium.com/subscribe
  3. You can support my work by buying me a cup of coffee on https://www.buymeacoffee.com/linuxshots

Thank You!

Navratan Lal Gupta

Linux Shots

Terraform
Open Source
Security
DevOps
Encryption

--

--

Navratan Lal Gupta
Linux Shots

Written by Navratan Lal Gupta

308 Followers
Editor for

I talk about Linux, DevOps, Kubernetes, Docker, opensource and Cloud technology. Don't forget to follow me and my publication linuxshots.

Help

Status

About

Careers

Press

Blog

Privacy

Terms

Text to speech

Teams