Environment variables with Google App Engine and CI/CD

When deploying your application via CD to a Google App instance you want to be able to pass along environment variables without setting them inside app.yaml file.

The only ‘official’ way to configure environment variables inside a Google App Engine is to hardcode them inside your app.yaml config file. Of course this is not something you want to do because you’ll probably want to have this file inside a VCS. This way you can automate your deployments with a CD tool like Bitbucket pipelines and easily setup multiple environments (DEV/STAG/PROD) in the cloud.

After searching on the internet I had to come up with a solution myself. It’s a small bash script that will become a deployment step inside your CD pipeline. It will basically create a regular .env file with environment variables defined inside your pipeline. This way I can configure for example a different database for DEV/STAG/PROD.

app-dev.yaml file

runtime: nodejs10

instance_class: F1
automatic_scaling:
  max_instances: 1

entrypoint: "node dist/main.js"

env_variables:
  # Database settings
  TYPEORM_MIGRATIONS_RUN: "true"
  TYPEORM_SYNC: "true"
  TYPEORM_LOGGING: "true"
  GOOGLE_STORAGE_BUCKET: "cf-storage-dev"

This is the yaml file that Google App Engine will use to configure your instances. You can setup multiple yaml files for each cloud environment (DEV/STAG/PROD). As you can see, you can specify environment variables here. I only configure static variables here that are allowed to be inside VCS.

generate-env-file.sh script

#!/bin/sh
# Declare all environment variables that have to be copied
declare env_array=(
TYPEORM_HOST
TYPEORM_DATABASE
TYPEORM_USERNAME
TYPEORM_PASSWORD
TYPEORM_DRIVER
TYPEORM_TYPE
APP_TOKEN_SECRET
)

# Set file output name
FILE_OUTPUT_NAME=.env

# Prevent overwriting an existing .env file
if [ -f ".env" ]; then
    echo "A .env file is already present in the current directory."
else
  # Print all environment variables to .env file
  for i in "${env_array[@]}";
  do
     printf '%s\n' "${i}=${!i}" >> $FILE_OUTPUT_NAME;
  done
fi

Create a file generate-env-file.sh in the root of your project. This bash script will create an .env file containing all the environment variables that are specified inside the env_array AND if they available inside the environment. In other words, you have to make sure you configure the same variables as defined inside env_array in your CD pipeline.

bitbucket configuration

In bitbucket, navigate to your repo. Then navigate to settings > pipelines > deployments. There you can configure environment variables your app environments (DEV/STAG/PROD).

bitbucket-pipelines.yml

# This is a sample build configuration for JavaScript.
# Check our guides at https://confluence.atlassian.com/x/14UWN for more examples.
# Only use spaces to indent your .yml configuration.
# -----
# You can specify a custom docker image from Docker Hub as your build environment.
image: node:10.15.3

pipelines:
  branches:
    development:
      - step:
          name: Build project via Yarn
          caches:
            - node
          script:
            - yarn install
            - yarn build
          artifacts:
            - dist/**
      - step:
          name: Deploy to Google Cloud
          deployment: Development
          script:
            - chmod +x ./generate-env-file.sh
            - bash ./generate-env-file.sh
            - cat ./.env
            - pipe: atlassian/google-app-engine-deploy:0.4.1
              variables:
                KEY_FILE: $KEY_FILE
                PROJECT: 'NAME_OF_GCP_PROJECT'
                DEPLOYABLES: 'app-dev.yaml'
                VERSION: '${BITBUCKET_BUILD_NUMBER}'
                EXTRA_ARGS: '--verbosity=debug'

Make sure to set the option “deployment” to match an environment name from your bitbucket deployments config! (see step above)

If you aren’t using bitbucket pipelines yet, you defiantly have to try it out. The file above will create a CD pipeline for the branch ‘development’ inside your repo. Basically every time you push to this branch this pipeline will be fired. In the first step we install all dependencies and build the project, in the second step our bash script is executed and the project is uploaded to Google App Engine, including the .env file. Sweet!