Google Cloud Run on Rails: a real life example (Part 3: production environment and security)

In Part 1 of this tutorial we setup our project environment and Part 2 was dedicated to running the Photo Album application in your local development environment.

Setting up Cloud Run

If not already done in Part 1, enable the Cloud Run API:

$ gcloud services enable run.googleapis.com

Also define the Google Cloud region Cloud Run will be running in:

$ gcloud config set run/region us-central1

Note: as of this writing Cloud Run is still in beta phase and, therefore, only available from the us-central1region

Setting up Rails master key

Since version 5.2, Rails generates a secret master key to encrypt user session parameters in cookies, other types of sensitive information going back and forth between the application and the browser as well as your own application data of need be. To create a fresh rails master key and the corresponding credentials files, type the following command:

$ EDITOR=vim rails credentials:edit # generate secret_key_base

It fires up the vi editor with the new credentials file already populated so you just have to save it. The encrypted version of the credentials file is in config/credentials.yml.enc and the corresponding master key in config/master.key. The later is a very sensitive piece of information that must never be accessed by unauthorized persons or programs. We’ll see in a moment how to protect the master key.

Setting up your production database with Cloud SQL

If not already done in Part 1, enable the Cloud SQL Admin API.

$ gcloud services enable sqladmin.googleapis.com

Cloud SQL, the managed version of MySQL available on GCP, is our RBDMS of choice for this tutorial.

Let’s create a so called ‘db-f1-micro’ server instance, the smallest Cloud SQL instance available. It is perfectly suited for our demo application (see other types of MySQL instances). We also need to protect the database root account with a password and define another account that we will be specifically used by Rails in production.

# Create a small Cloud SQL instance with a public IP (takes some time)
$ gcloud sql instances create photo-album-production --tier=db-f1-micro --region=us-central1 --assign-ip
# Protect database root account
$ gcloud sql users set-password root --host % --instance photo-album-production --password your_root_db_password
# Create a new database account for Rails in production
$ gcloud sql users create prod_db_user --instance photo-album-production --host % --password your_prod_db_password

Let’s check that our Cloud SQL instance is ready to use. The command below should show your instance as “RUNNABLE”.

$ gcloud sql instances list

Visit your Cloud SQL instance on the Google Cloud Web console, copy the instance connection name and paste it into the socket field of the production section in config/database.yml .

production:
  <<: *default
  database: photo_album_production
  username: prod_db_user
  password: <%= ENV[‘DATABASE_PASSWORD’] %>
  socket: “/cloudsql/project_id:us-central1:photo-album-production”

Security notes: you probably already know that storing passwords and credentials in clear in the source code of your application is an absolute NO-NO from a security standpoint. This is why for production deployment Rails will get its database password at runtime from the DATABASE_PASSWORD environment variable. Later in this article, Google Cloud Key Management System (Cloud KMS) will help us encrypt this password and decrypt it only when Rails actually needs it.

Creating the image bucket on GCS

The Photo Album application relies on Rails Active Storage to manage image files. Active Storage is a really cool module that facilitates uploading files of any type and storing those files on a choice of storage backends. When running locally in the development or test environment the storage backend is usually your local disk. In production we chose to use Google Cloud Storage (GCS) as our backend for maximum scalability and reliability.

Let’s create the GCS bucket to store images in the same location as the Cloud Run instance for best performance:

$ gsutil mb -p $PROJECT_ID -l us-central1 gs://photo_album_images_xxxxxx

Where xxxxxx is a 6 digit number of your choice to make for a unique bucket name.

Then let’s point Active Storage to this bucket by editing the config/storage.yml file. Look for the google section and adjust the project parameter with your own project ID and the bucket name.

google:
  service: GCS
  project: project_id
  credentials: <%= Rails.root.join(“config/photo_album_runner.key”) %>
  bucket: photo_album_images_xxxxxx

Also edit the config/environments/production.rb file to inform Active Storage that we will be using the Google Storage backend in production:

config.active_storage.service = :google

Creating the application Service Account

Before we can deploy our application in production and run it for the first time we must associate it with a service account. In brief, a service account is a special Google account that impersonates your application whenever it interfaces with other GCP services.

Let’s create a service account called “photo-album-runner” with Google Cloud IAM (Identity & Access Management):

$ gcloud iam service-accounts create photo-album-runner --display-name “Photo Album Cloud Runner”

See what the full service account name is with the following command:

$ gcloud iam service-accounts list
NAME                     EMAIL                             DISABLED
Photo Album Cloud Runner photo-album-runner@photo-album-xxxxxx.iam.gserviceaccount.com False
...
...

Note: you will probably also notice 2 other service accounts in the list. Those are automatically created by GCP whenever you create a new project : one Compute Engine service account and one App Engine service account. We won’t be using any of those for this tutorial.

The next step is to grant roles to our brand new service account so that it gets enough latitude to perform its duties. So what does that service account needs to be doing exactly?

• First it needs to view, create, delete objects in the GCS bucket as this is where we will be storing images. This can be achieved by granting the storage.objectAdmin role.
• It must also be able to talk to our Cloud SQL instance as a client (cloudsql.clientrole).

The photo-album-runner service account is now equipped with exactly the roles it needs. No more, no less.

The next step is to ask Cloud IAM to generate a service account key for us in the form of a JSON file saved in the config/photo_album_runner.key file. This file will serve as Google credentials at runtime for the Photo Album application to establish its identity with other GCP services.

$ gcloud iam service-accounts keys create ./config/photo_album_runner.key --iam-account photo-album-runner@$PROJECT_ID.iam.gserviceaccount.com

This credential file is definitely a sensitive piece of information. Should someone gain access to this file, s/he could too access to the various GCP services we are using. It’s quite clear that this key file as well as other sensitive pieces of information of our application must absolutely be protected. Google Cloud KMS to the rescue!

Keeping our secrets… secret

Fortunately Google Cloud Key Management Service (Cloud KMS) is precisely intended to deliver that kind of service and it runs gracefully with Google Cloud Build. With this in mind, let’s step back for a moment and make a list of all the resources in our Rails application that require encryption:

• The master key file generated by Rails (config/master.key) as explained above.
• The Rails production database password that we will pass in the DATABASE_PASSWORD environment variable at runtime (see config/database.yml)
• The credentials of the GCP service account that impersonates our application when running in Cloud Run

Since Rails 5.2 all encrypted secrets, those needed by Rails and possibly other secrets of your application, are stored in a single encrypted file: config/credentials.yml.enc. The master key used to encrypt/decrypt this file is in config/master.key. You will notice that Rails automatically adds the master key file name to .gitignore to make sure it is never committed to your source repository for obvious security reasons.

Since this file is not part of our source repository we must find another secure way of providing Rails with this master key file as it is both needed to build the application container and subsequently to run the Rails application in the container. Let’s see how to do that.

If not already done in Part 1, enable the Google Cloud KMS API from the Web console (screenshot below) or from the command line:

$ gcloud services enable cloudkms.googleapis.com

Enabling Cloud KMS for your project from the Google Cloud Web Console

Let’s now create a key ring named photo-album on which to hook our secret keys, one for the Photo Album service account and one for the Rails master key.

The two encrypted files config/photo_album_runner.key.enc and config/master.key.enc can safely be commited to your source repository. Later when building the container image, Cloud Build will ask Cloud KMS to decrypt the secret keys but only when needed and only in a context that cannot be accessed by anyone, therefore protecting your secrets.

Let’s also encrypt the database password that we will be using for the production database so that we avoid leaving the production password in clear in any of our source files (typically config/database.yml or Dockerfile ). Replace xxx below with the password you chose when creating the prod_db_user earlier in the Cloud SQL section:

$ gcloud kms keys create db_pwd_key --location=us-central1 --keyring photo-album --purpose encryption
$ echo -n “xxx” | gcloud kms encrypt --location us-central1 --keyring photo-album --key db_pwd_key --plaintext-file - --ciphertext-file -| base64 --wrap 0

Copy-paste the base64 encoded cypher in the cloudbuild.yaml file (see the DB_PWD entry at the end of the file)

This is the end of Part 3. Our production environment is all set. Part 4 (and the last for now) of this tutorial addresses building the container image and deploying it in production.