Encrypt files on S3 with ActiveStorage

When it comes to storing files using ActiveStorage on a S3 Bucket, nothing is easier.

You just have to:

• create your S3 bucket
• use the “aws-sdk-s3”
• specify inside your config/storage.yml the S3 Service with your bucket’s name
• And then ActiveStorage, with the aws-sdk-s3, is taking care of the rest.

But when it comes to storing sensitive informations that your users upload on your app, you’d want to take extra care of how you store it. Even though S3 is using by default a layer of encryption to encrypt the keys of the files, the files are themselves stored clearly inside the bucket.

What if some day, for some reasons, your S3 bucket has leaked, and all the sensitive files of your users fall under the hands of malicious persons ? That’s something you’d surely want to avoid.

That’s why we decided to encrypt all the sensitive files on the server’s side at the documents upload, and decrypt a file through a dedicated controller when we want to access it.

Step 1 — Create a EncryptedS3Service

To be able to store a file on S3 through ActiveStorage, you’ll need the “aws-sdk-s3”, so add this line to your Gemfile

gem "aws-sdk-s3", "~> 1.134.0", require: false

Then, you’ll have to write a custom S3Service using the Aws::S3::EncryptionV2::Client, in a lib/active_storage/service folder.

This service will take all the necessary arguments to encrypt/decrypt a file before uploading/downloading it.

In our case, we decided to have an encryption_key stored in our AWS account, but you can perfectly use KMS by passing a kms_key_id.

## lib/active_storage/service/encrypted_s3_service.rb

require "active_storage/service/s3_service"

module ActiveStorage
  class Service
    class EncryptedS3Service < Service::S3Service

      attr_reader :encryption_client

      KEYS_OPTIONS = {
        key_wrap_schema: :aes_gcm,
        content_encryption_schema: :aes_gcm_no_padding,
        security_profile: :v2, # rubocop:disable Naming/VariableNumber
      }.freeze

      def initialize(bucket:, upload: {}, **options)
        super_options = options.except(:kms_key_id, :encryption_key)
        super(bucket:, upload:, **super_options)
        @encryption_client = Aws::S3::EncryptionV2::Client.new(options.merge(KEYS_OPTIONS))
      end

      ## more to come here...
    end
  end
end

Let’s dive in what’s going on here !
The KEY_OPTIONS constant will add all the encryption options that the Aws::S3::EncryptionV2::Client needs. Here’s are the possible values to use:

• key_wrap_schema -> the key used to wrap the schema, which should match the type of key used. In our case, :aes_gcm !
• content_encryption_schema -> :aes_gcm_no_padding is the only option available. Given what is specified in the documentation, more options will be available in future releases.
• security_profile -> if you’ve never encrypted files with Aws::S3::Encryption::Client, you’d better use :v2 argument, otherwise, you could use :v2_and_legacy

We then initialize a Aws::S3::EncryptionV2::Client with the S3Service native options + the KEY_OPTIONS.

Finally, we’ll override uploads and downloads method to encrypt/decrypt a file.

## lib/active_storage/service/encrypted_s3_service.rb
require "active_storage/service/s3_service"

module ActiveStorage
  class Service
    class EncryptedS3Service < Service::S3Service

      ### …initialization code…

     def upload(key, io, checksum: nil, **)
        instrument(:upload, key:, checksum:) do
          encryption_client.put_object(
            upload_options.merge(
              body: io,
              bucket: bucket.name,
              key:
            )
          )
        rescue Aws::S3::Errors::BadDigest
          raise ActiveStorage::IntegrityError
        end
      end

      def download(key)
        raise NotImplementedError, "#get_object with :range not supported yet" if block_given?

        instrument(:download, key:) do
          encryption_client.get_object(
            bucket: bucket.name,
            key:
          ).body.string.force_encoding(Encoding::BINARY)
        end
      end

      def download_chunk(key, range)
        raise NotImplementedError, "#get_object with :range not supported yet"
      end

    end
  end
end

Step2 — Call the EncryptedS3Service inside your config/storage.yml

You now want to define all the options inside you config/storage.yml.

To do so, you’ll need:

• a bucket name
• the region where your bucket is
• an encryption_key (or a kms_key_id)

Your storage.yml should look like this:

encrypted_active_storage:
  service: EncryptedS3
  bucket: your-bucket-name
  region: eu-west-3
  encryption_key: ENV["your-securely-stored-encryption-key"]

Then when it comes to call this EncryptedS3Service, you’ll specify inside your model what service you need to use.

class MyModel < ::ApplicationRecord
   has_many_attached :id_documents, service: :encrypted_active_storage
end

And voila! You can now upload an encrypted file on S3.

Step3 — Download the file

To download an encrypted file, there’s a few gotchas that you have to keep in mind.

You’d naturally assume that inside your view, you’ll use the method rails_blob_path / rails_blob_url

<%= 
 link_to("My Encrypted File",
 rails_blob_path(my_model.my_file, disposition: "attachment"),
 target= "_blank"
 )
%>

This method will “successfuly” download a file, but the file will be corrupted, so it doesn’t properly work since this helper method automatically pass a block to stream the download and yield the blob in chunks.

Remember, the file has been encrypted as a whole, so the decryption should be as well on the whole file.

The reason for that is that downloading large files will use a lot of RAM, and since the encryption/decryption is expensive, you should wisely decide what files should be encrypted or not with this kind of pattern.
Anyway, we decided to bypass this behavior by creating our own routes to download an encrypted file.

First, you’ll need to define your route

# config/routes.rb
Rails.application.routes.draw do
   # some routes…
   resources :decrypted_blobs, only: :show
end

Second, you’ll have to create its associated controller

class DecryptedBlobsController < BaseController

    def show
      attachment = ::ActiveStorage::Attachment.find(params[:id])

      decrypted_io = attachment.download

      send_data decrypted_io,
        filename: attachment.filename.to_s,
        type: attachment.content_type,
        disposition: "attachment"
    rescue ActiveStorage::FileNotFoundError
      redirect_back(fallback_location: backoffice_home_path, flash: { alert: "Something unexpected happened" })
    end

end

And then you can call your new decrypted_blob_url like this

<%= link_to(attachment.filename, decrypted_blob_url(attachment, attachment.id)) %>

So that’s it, you can now encrypt and decrypt files stored on AWS S3 by using ActiveStorage.

One caveat to keep in mind: this method uses server-side uploads, which can be costly in backend resources.

Consider mixing it with client-side uploads or even client-side encryption if you expect a high volume of uploads.

We also thought of handling the encryptions by chunks instead of encrypting the whole file, and then decrypting those chunks, to not overload the server by download the whole file into memory.

But this article was written because it lacked some contexts and documentations on the internet about this matter, and using ActiveStorage is simple, so encryption should be simple as well.

Finally, this article has been widely inspired by this article wrote by Andrew Kane. Unfortunately, the Aws::S3::Encryption::Client has been deprecated by AWS, so we had to freshen up its tutorial a bit !