Docker & ENV: Methods for Passing Variables through Docker-Compose

Passing environment variables into your Docker build process and containers can be tricky and downright frustrating at times, especially if you want to automate building and deploying multiple containers using docker-compose. However, there are many situations where this is required for security and ease of development and deployment.

At our startup, Blockade Games, we are building multiple services and components that may need to talk to each other, but are maintained in independent repositories for proper separation of concerns and ease of swapping compatible components. The major drawback to this approach is that building, configuring, and deploying multiple services becomes a logistical nightmare for developers. To address this, we’ve turned to Docker and Docker Compose to simplify as much of the process into two steps: build and deploy.

In order to keep things as secure and simple as possible, we need to spend time writing and refining each Dockerfile and docker-compose.yml to reduce the final build, configure, and deploy actions to one step each. A big part of making this possible is to pass in configuration variables to any part of the process.

Why do we want to do this?

Two good reasons come to mind: the first is for ease of cross-environment deployment, and the second is to avoid leaking sensitive information, notably authentication credentials.

When deploying code across multiple developer environments, there may be some machine- or user-specific configurations that should be set during build or prior to runtime. You can store configurations in the repo for each developer, but that becomes cumbersome and a potential security issue. Instead, it can make more sense to provide an interface for inserting these configurations.

A common security vulnerability is to accidentally or intentionally include keys, authentication tokens, or other authentication credentials within source code repositories to simplify the process of building and deploying code for development or production. Even if the source code is private, the publication of any authentication information can be leaked to adversaries and exploited.

Common ways to leak this information include making formerly private repos public without revoking the keys or authentication tokens beforehand, leaving source code exposed in any insecure development or deployment environments, or if any of the users with access to the repositories are compromised in any way. The best way to prevent this issue is to never include authentication details within the source repository, and in cases where this is done, to restrict the access for the authentication credentials to mitigate the possible damage that leaking the information can cause.

Let’s dive into the different ways to pass variables into Docker containers, Dockerfiles, and Docker Compose:

Pass variables into Docker Compose during the `build` or `up` process

VAR=VALUE docker-compose build
VAR=VALUE docker-compose up

The above variable, VAR, becomes visible in the docker-compose.yml file for both build and up commands.

Note: An alternative to this is to provide a .env file in the directory where the docker-compose.yml file lives, however this method can expose the environment variables through the source code repository.

Example snippet from docker-compose.yml:

version: '3.7'
services:
 server:
  build:
    context: ${SERVER_DIRECTORY} # build from another directory
    dockerfile: docker/dev/Dockerfile
  image: server-dev:latest
  container_name: server-dev
  restart: always
  environment:
    DATABASE_HOST: ${SERVER_DB_HOST}
    DATABASE_PORT: ${SERVER_DB_PORT}
    DATABASE_DB: ${SERVER_DB_NAME}
    DATABASE_USER: ${SERVER_DB_USER}
    DATABASE_PASS: ${SERVER_DB_PASS}
  volumes:
    - ${SERVER_DIRECTORY}:/usr/src/app/

Pass variables into Dockerfile through Docker Compose during build

docker-compose build --build-arg GIT_PERSONAL_ACCESS_TOKEN="{TOKEN}"

If you want to pass variables through the docker-compose process into any of the Dockerfiles present within docker-compose.yml, use the --build-arg parameter for each argument to flow into all of the Dockerfiles.

Example Dockerfile, which is referenced in docker-compose.yml:

FROM node:10
# Create our git credentials
ARG GIT_PERSONAL_ACCESS_TOKEN
RUN echo "https://${GIT_PERSONAL_ACCESS_TOKEN}@github.com" > /root/.git-credentials
WORKDIR /usr/src/app
VOLUME /usr/src/app
CMD npm install && npm start

Pass variables into Docker containers at runtime, when bringing them up

To pass variables from docker-compose up into the running Docker containers, you can use a mix of the above techniques. You can pass in environment variables into docker-compose using the first technique, and expose them through the environment configuration, or you can pass them into the Dockerfile during build and expose them as environment variables.

What this lets you do

By passing in configuration variables and authentication credentials, you can leave this information out of the repository and securely distribute and store authentication tokens without concern that you’re leaking keys through the source repository.

Some things we’ve done with these methods include:

• Providing our git-credentials so that we can automate pulling down private repositories without needing to make them public or distributing Personal Access Tokens through the repository
• Build multiple Docker containers at once from a single docker-compose.yml without needing to assume the locations of the other repositories (which also lets us avoid using submodules inappropriately)
• Create personalized .env files during the Dockerfile build process with the settings specific to a given environment, user, or application

More Information

A great resource and rundown on the ENV and ARG parameters for Dockerfiles: https://vsupalov.com/docker-arg-env-variable-guide/