Release a Phoenix application with docker and Postgres

9 min readDec 31, 2019

Source https://unsplash.com/photos/9cCeS9Sg6nU

This is a minimal introduction on how to bundle and release a Phoenix web application accessing a Postgres database with Docker and the new released feature introduced in Elixir 1.9. We will also create a custom entrypoint which waits for the database to boot and sets the database up with the migrations from Ecto. Finally we will be able to start the complete application with a single docker-compose up command and everything will be ready to use.

How to use this guide

For all code snippets executed in the command line, the current folder is shown before the dollar sign. E.g.:

~$ pwd

Will produce /home/$user

Make sure to remove the # File comments for each code snippet you copy and paste into your files. They are not intended to be in the real file and are only there to provide clarity.

Prerequisites

To follow this guide, make sure you have at least the following applications installed on your machine. In the brackets are the versions with which this guide was written.

Elixir (1.9.4 — Setup)
Erlang (OTP 22 — erts-10.6)
Docker (19.03.3 — Debian Setup)
Node.js with npm (12.13.0 — Unix Setup)
Phoenix (1.4.10 — Setup)

Make sure to install at least Elixir 1.9 since we are going to use the newly introduced release feature.

Creating a new blank phoenix application

To create a new phoenix application we can use the following mix command. Replace my_app with whatever you want to name your application.

~$ mix phx.new my_app

Go ahead and fetch and install all the dependencies. Now we created a blank phoenix project. Now before we are able to test our server we need to start a new Postgres database.

~$ docker run --name postgres \
    -e POSTGRES_PASSWORD=postgres \
    -p 5432:5432 -d --rm postgres

Once the Postgres server is up and running, we need to create the database. For this task we will use the Ecto toolkit to create the database. Therefore, we switch into the project directory and setup the database with the following commands.

~$ cd my_app
my_app$ mix ecto.create

In order to verify that everything was set up correctly we start the development server.

my_app$ mix phx.server

If you got no error messages, you should see the following page if you navigate the browser to localhost:4000.

Running Phoenix application at localhost:4000

Application configuration

If our phoenix application is running in a docker container, it should be possible to provide runtime information like the parameters of the database connection when the container is started. We will inject this configuration using environment variables. This way it is easy to change these parameters inside the docker-compose.yml file or by using the --env flag when using the docker run command.

Elixir provides two mechanisms when working with application configuration in the new releases feature.

Build-time configuration

Whenever the mix command is used to run a project, it looks for the config/config.exs file and usually will also import another config file, like config/dev.exs, depending on the value of MIX_ENV. To bundle a release we will use the mix release command. Therefore, all configuration assignments in these config files will be set at compile time, which will be the moment we build our docker image. That means that all assignments which happen to be in these config files can’t be changed when running the docker image.

# File: config/config.exs
use Mix.Configconfig :my_app, :secret_key, System.fetch_env!("MY_APP_SECRET_KEY")

The config above will be executed at build time, which means that each time we start the docker image, we will not be able to set the :secret_key with the MY_APP_SECRET_KEY environment variable.

Runtime configuration

Because we want to be able to set runtime configuration, the Elixir team introduced another way to inject environment variables into our application. The minimal config file config/releases.exs looks like the following:

# File: config/releases.exs
import Configconfig :my_app, :secret_key, System.fetch_env!("MY_APP_SECRET_KEY")

Now we are able to set the MY_APP_SECRET_KEY each time we start the docker image.

Your config/releases.exs file needs to follow three important rules:

It MUST import Config at the top instead of the deprecated use Mix.Config
It MUST NOT import any other configuration file via import_file
It MUST NOT access Mix in any way, as Mix is a build tool and it not available inside releases

Source

Build a release

Now that we can distinguish when to use which mechanism to load our application configuration we can split the config/prod.exs and config/prod.secret.exs files into build-time and runtime configuration files.

Due to the fact that we will read our runtime configuration with the config/releases.exs file we can safely delete the config/prod.secret.exs file. Go ahead and delete it now and replace the contents of the config/prod.exs file with the following.

# File: my_app/config/prod.exs
use Mix.Config# Used to generate urls
config :my_app, MyAppWeb.Endpoint,
  url: [host: "example.com", port: 80],
  cache_static_manifest: "priv/static/cache_manifest.json"# Do not print debug messages in production
config :logger, level: :info# ## Using releases (Elixir v1.9+)
#
# If you are doing OTP releases, you need to instruct Phoenix
# to start each relevant endpoint:
#
config :my_app, MyAppWeb.Endpoint, server: true

For demonstration purpose we will be able to set our database configuration with environment variables. All but the database host parameter will have a default value. Create the config/releases.exs file and insert the content below.

# File: my_app/config/releases.exs
import Configdb_host = System.get_env("DATABASE_HOST") || 
  raise """
  environment variable DATABASE_HOST is missing.
  """
db_database = System.get_env("DATABASE_DB") || "my_app_dev" 
db_username = System.get_env("DATABASE_USER") || "postgres" 
db_password = System.get_env("DATABASE_PASSWORD") || "postgres"
db_url = "ecto://#{db_username}:#{db_password}@#{db_host}/#{db_database}"config :my_app, MyApp.Repo,
  url:  db_url,
  pool_size: String.to_integer(System.get_env("POOL_SIZE") || "10")secret_key_base = System.get_env("SECRET_KEY_BASE") ||
  raise """
  environment variable SECRET_KEY_BASE is missing.
  You can generate one by calling: mix phx.gen.secret
  """ 
config :my_app, MyAppWeb.Endpoint,
  http: [:inet6, port: 4000],
  secret_key_base: secret_key_base

Since I personally use Traefik for all my running containers, I don’t care about the configuration of the port, so I hardcoded it. We also use the development database as a default value, because we know that this database is already set up.

At this point we can test the created config files by creating a release on our local machine.

my_app$ MIX_ENV=prod mix release
my_app$ _build/prod/rel/my_app/bin/my_app start

Starting our release will successfully fail. That just means that our configuration actually works because we raise an exception if System.get_env("DATABASE_HOST") returns nil. If we provide all required parameters the server should start.

my_app$ mix phx.gen.secret
n12...long-secret...BFyDl
my_app$ export SECRET_KEY_BASE=n12...long-secret...BFyDl
my_app$ export DATABASE_HOST=localhost
my_app$ _build/prod/rel/my_app/bin/my_app start

Make sure your Postgres database is still running and your default password is correctly set in the config/releases.exs file. If you get the “database doesn’t exist error”, rerun the mix ecto.setup command and check if your default database is correctly set.

Now we should see our application running at localhost:4000. We can ignore the static manifest error at this point.

Dockerize all the things!

The Dockerfile we are going to use is an adjusted version from the official Phoenix guide. We are using a multi-stage build to make the smallest possible image.

# File: my_app/Dockerfile
FROM elixir:1.9-alpine as build

# install build dependencies
RUN apk add --update git build-base nodejs npm yarn python

RUN mkdir /app
WORKDIR /app

# install Hex + Rebar
RUN mix do local.hex --force, local.rebar --force

# set build ENV
ENV MIX_ENV=prod

# install mix dependencies
COPY mix.exs mix.lock ./
COPY config config
RUN mix deps.get --only $MIX_ENV
RUN mix deps.compile

# build assets
COPY assets assets
RUN cd assets && npm install && npm run deploy
RUN mix phx.digest

# build project
COPY priv priv
COPY lib lib
RUN mix compile

# build release
# at this point we should copy the rel directory but
# we are not using it so we can omit it
# COPY rel rel
RUN mix release

# prepare release image
FROM alpine:3.9 AS app

# install runtime dependencies
RUN apk add --update bash openssl postgresql-client

EXPOSE 4000
ENV MIX_ENV=prod

# prepare app directory
RUN mkdir /app
WORKDIR /app

# copy release to app container
COPY --from=build /app/_build/prod/rel/my_app .
COPY entrypoint.sh .
RUN chown -R nobody: /app
USER nobody

ENV HOME=/app
CMD ["bash", "/app/entrypoint.sh"]

The order of these commands is indeed important. Docker will try to cache as many steps as possible. So if you change files in the lib directory, it will use the cache up to line COPY lib lib and start the new build from there.

The main difference is that we are going to use a custom entry point script to start our application. The entrypoint.sh script will make sure that the Postgres server has finished starting and that the database is correctly setup.

# File: my_app/entrypoint.sh
#!/bin/bash
# docker entrypoint script.

# assign a default for the database_user
DB_USER=${DATABASE_USER:-postgres}

# wait until Postgres is ready
while ! pg_isready -q -h $DATABASE_HOST -p 5432 -U $DB_USER
do
  echo "$(date) - waiting for database to start"
  sleep 2
done

bin="/app/bin/my_app"
# start the elixir application
exec "$bin" "start"

This is a good point to test if the configuration up to this point is working.

my_app$ chmod +x entrypoint.sh
my_app$ docker build -t my_app .
my_app$ docker run --rm \
  -e DATABASE_HOST=localhost -e SECRET_KEY_BASE=1 \
  --net=host my_app

For testing purposes we will use the value “1” for our SECRET_KEY_BASE. Don’t use this value in your real world applications 😉. To access our database hosted in the other running container, we will just use the --net=host option. That way our new container will be linked to the host network. Since docker already forwards 0.0.0.0:5432 (will match localhost:5432) to our database container this will work just fine. To finally test if our entrypoint.sh script actually works we will need to stop the database, start our app and then again start our database.

my_app$ docker stop postgres
my_app$ docker run --rm \
  -e DATABASE_HOST=localhost -e SECRET_KEY_BASE=1 \
  --net=host my_app

This should print the “waiting for database to start” message. Due to the fact that we didn’t start our app container in detached mode, we need to open another terminal window and start the database again. If you used the exact same command to start your database as we used in the first chapter of this guide, the database container will be removed when you stop it. Hence we cannot start the container with a simple docker start postgres command. Therefore, we will spin up a new postgres container.

~$ docker run --name postgres \
  -e POSTGRES_PASSWORD=postgres \
  -p 5432:5432 -d --rm postgres
~$ cd my_app
my_app$ mix ecto.setup

Again we should see our application running at localhost:4000. At this point we have a working docker image. In the next chapter we will have a look at how to extend our entrypoint.sh script to setup the database for us and how the final docker-compose.yml file looks like.

Completing the production setup

The final part of this guide is about finishing up our production setup with docker-compose. We use docker-compose to start a database and the application itself. If we start the database with docker-compose we don’t need to access it from anywhere else but our application. Therefore, we will make sure that only the application will be able to access it with isolated network.

If only the application can access the database, we cannot setup our database with the convenient mix ecto.setup command. To solve this issue we will need to adapt our entrypoint.sh script.

# File: my_app/entrypoint.sh 
...
bin="/app/bin/my_app"
eval "$bin eval \"MyApp.Release.migrate\""
# start the elixir application
exec "$bin" "start"

Before we start the application with the exec command on line 6, we will insert the eval "$bin eval \"MyApp.Release.migrate\"" command. This will run the database setup with our migrations. Now we need to create the my_app/lib/my_app/release.ex migration file.

# File: my_app/lib/my_app/release.ex
defmodule MyApp.Release do
  @app :my_app

  def migrate do
    for repo <- repos() do
      {:ok, _, _} = Ecto.Migrator.with_repo(repo, &Ecto.Migrator.run(&1, :up, all: true))
    end
  end

  def rollback(repo, version) do
    {:ok, _, _} = Ecto.Migrator.with_repo(repo, &Ecto.Migrator.run(&1, :down, to: version))
  end

  defp repos do
    Application.load(@app)
    Application.fetch_env!(@app, :ecto_repos)
  end
end

This is a generic setup provided by the official Phoenix guide. There is only the first and second line that needs to be changed if your project has a different name. The final missing part is the docker-compose.yml file itself. Therefore, we create it and fill the following content in.

# File my_app/docker-compose.yml
version: "3"

networks:
  internal:
    external: false

services:
  app:
    image: my_app:0.1.0
    build:
      context: .
    environment:
      - SECRET_KEY_BASE=${SECRET_KEY_BASE}
      - DATABASE_DB=${DATABASE_DB}
      - DATABASE_HOST=db
    ports:
      - 4000:4000
    networks:
      - internal
    depends_on:
      - db

  db:
    image: postgres:9.6
    volumes:
      - ./data/db:/var/lib/postgresql/data
    environment:
      - POSTGRES_PASSWORD=postgres
      - POSTGRES_DB=${DATABASE_DB}
    networks:
      - internal

The docker-compose.yml file creates an internal network so that the application can access the database through an isolated network. By using the ${} notation in our file we are able to inject environment variables. To set these variables we can use either a .env file or the system environment variables set by the export command. Docker-compose will automatically check if a .env file exists in the current directory. If you want you can also directly set your variables inside the docker-compose.yml file.

Finally we can start our complete application isolated with a single command 🥳🥳🥳

my_app$ docker-compose up

You may need to export the required environment variables first.

my_app$ mix phx.gen.secret
n12...long-secret...BFyDl
my_app$ export SECRET_KEY_BASE=n12...long-secret...BFyDl
my_app$ export DATABASE_HOST=localhost

That was a very minimalistic approach to deploy your Phoenix application with docker-compose. You can find the full code in my Github repository.