How to Securely Encrypt Large Files and Upload to AWS S3 Using AWS KMS and OpenSSL

In today’s digital age, data security is more critical than ever. Whether you’re handling sensitive business documents, personal data, or configuration files, ensuring that your data is encrypted and securely stored is paramount. This article walks you through a powerful script leveraging AWS Key Management Service (KMS) and OpenSSL to encrypt and decrypt large files, ensuring your data stays safe and sound in AWS S3.

Why Do You Need This Script?

Storing sensitive data in plain text is risky. Unauthorized access to such data can lead to severe security breaches and loss of sensitive information. Encrypting your files before storing them in a cloud storage service like Amazon S3 ensures that the data remains secure even if the storage is compromised. This script provides an efficient and automated way to handle encryption and decryption of your files using AWS KMS and OpenSSL.

The Script: A Step-by-Step Guide

Let’s dive into the script and understand how it works. The script is designed to perform two primary functions: encrypting and decrypting files. We use AWS KMS to generate encryption keys and OpenSSL to perform the encryption and decryption.

Step-by-Step Breakdown

Step 1: Setting Up the Environment

The script starts by setting the LC_ALL environment variable to C to handle byte sequences correctly across different locales. This ensures that commands like tr and base64 behave consistently.

Step 2: Encryption Function

The encrypt function performs the following tasks:

1. Environment Variable Check: Ensures that KMS_KEY_ID and S3_BUCKET_NAME are set.
2. Generate DEK: Calls aws kms generate-data-key to create a Data Encryption Key (DEK) with both plaintext and ciphertext representations.
3. Extract Keys: Uses jq to parse the JSON output and extract the plaintext and ciphertext keys.
4. Encrypt the File: Uses OpenSSL to encrypt the file with the plaintext DEK.
5. Save and Upload Keys: Saves the ciphertext DEK to a file and uploads both the encrypted file and the DEK to an S3 bucket.
6. Clean Up: Securely deletes the plaintext DEK to prevent any potential security risks.

Step 3: Decryption Function

The decrypt function handles the decryption:

1. Environment Variable Check: Ensures that KMS_KEY_ID, S3_BUCKET_NAME, and DATE are set.
2. Download Files: Downloads the encrypted file and the DEK from the S3 bucket.
3. Verify Downloads: Checks if the files were downloaded successfully.
4. Decrypt the DEK: Uses aws kms decrypt to obtain the plaintext DEK.
5. Decrypt the File: Uses OpenSSL to decrypt the file with the plaintext DEK.
6. Clean Up: Securely deletes the plaintext DEK and the downloaded DEK file.

Usage

To use this script, run the following commands:

Encrypt the File:

./secure_backup.sh encrypt

Decrypt the File:

export DATE="YYYY-MM-DD" # the date represent the S3 path for the file to download and decrypt
./secure_backup.sh decrypt

Ensure that the required environment variables (KMS_KEY_ID, S3_BUCKET_NAME, and for decryption, DATE) are set before running the script.

References

For more details on AWS KMS and OpenSSL, refer to the official documentation:

• https://docs.aws.amazon.com/kms/latest/developerguide/overview.html
• https://www.openssl.org/docs/

By following this guide, you can ensure that your sensitive files are encrypted and securely stored in AWS S3, protecting your data from unauthorized access.