Mastering Elixir Releases with Distillery — A (Pretty) Complete Guide

How to create Elixir Releases. And some tips for Phoenix, Ecto, and umbrella applications.

A Quick Refresher and a Bit of History

Releases have long been around in the Erlang/OTP world.

I have previously written about what Releases are and why they’re great in more detail. Here is a little refresher: Releases are self-contained bundles of compiled Elixir/Erlang code that can easily be deployed on remote machines.

Naturally, most tools for creating Releases have so far come from the Erlang world. Erlang ships with systools and reltool, but their usage is rather complicated. So the community built relx as an easier-to-use alternative that has now become the de-facto standard for creating Releases from Erlang projects. Since those tools are all made to work with Erlang, they can work with Elixir projects but leave much to be desired.

In 2014, Paul aka bitwalker set out to make Elixir Releases easier and published exrm — built on top of relx with some Elixir-specific features. Due to limitations of relx, exrm was inflexible and never able to leverage the full potential of modular Elixir applications. So Paul decided to create a new Release manager, written from scratch in pure Elixir — and so Distillery was born.

Step-by-step guide

In this section of the article, you’ll learn how to add Distillery to a project and build your first release step-by-step. Let’s go!

I have created a small application that prints out the current time every second. We can use it to build a Release:

git clone https://github.com/wmnnd/distillery-demo.git
cd distillery-demo

Installing Distillery & Creating the First Release

Before you can create your first Release, you need to install Distillery. Add {:distillery, "~> 1.5"} to the list of dependencies in your project’s mix.exs and run mix deps.get.

Next, we initialize Distillery by calling mix release.init. This command creates a configuration file rel/config.exs. You could also manually create the file but it’s more convenient like this.

We will look at the contents of the configuration file in a bit. For now it’s enough to build our first Release. In order to do that, run mix release. Distillery will now create an output that looks something like this:

==> Assembling release..
==> Building release clock:0.1.0 using environment dev
==> You have set dev_mode to true, skipping archival phase
==> Release successfully built!
    You can run it in one of the following ways:
      Interactive: _build/dev/rel/clock/bin/clock console
      Foreground: _build/dev/rel/clock/bin/clock foreground
      Daemon: _build/dev/rel/clock/bin/clock start

Congratulations, you have just created your first Release! Run it with _build/dev/rel/clock/bin/clock foreground. As expected, the application now prints out the current time in one-second intervals. You can terminate the application by hitting Ctrl+C.

Note: There is currently a bug in Distillery that might crash the Release when starting it. In this case, run mix clean && mix compile and try running the application again.

In the call to start the application, you passed the foreground command at the end. This is the boot command. There are also other boot commands for starting an application: console launches an interactive iex session with your application and start creates a daemon process in the background.

Release Profiles and Environments

Let’s take a step back. You’re probably already familiar with Elixir’s build tool Mix. Mix supports multiple environments: dev, prod and test. Depending on the current environment, Mix can, for example, determine which dependencies or configuration files to include.

You might have noticed that when we created the Release, Distillery printed out this line: Building release clock:0.1.0 using environment dev. Distillery also has environments — but they are different from Mix environments. Distillery uses environments to allow for multiple configurations of the Release building process.

Conveniently — or confusingly — Distillery environments are by default the same as the Mix environments. Let’s take a look at the auto-generated configuration file rel/config.exs in order to better understand how profiles work:

use Mix.Releases.Config,
  default_release: :default,
  default_environment: Mix.env()

We can see that default_environment is set to Mix.env(). This means that Distillery will match our Mix environment. You can override default environment with the --env switch. Here are some examples of how this could work:

mix release
#Mix environment and Distillery environment are both dev
MIX_ENV=prod mix release
#Mix environment and Distillery environment are both prod
MIX_ENV=prod mix release --env=dev
#Mix environment is prod, Distillery environment is dev

The configuration file also specifies that default_release is :default. This is because you can actually define more than just one Release for your project. With the :default setting, Distillery automatically picks the first Release defined in rel/config.exs. If you want to override the default Release, you can specify this with the --name flag.

The combination of an environment with a specific Release name is called a profile. You can pick a specific profile with the --profile=name:env flag.

In the next section we’re going to take a look at how we can use multiple environments to configure Distillery.

Release Build Configuration

Our Distillery configuration includes two environments (dev and prod) on top of one Release named like our project (clock):

environment :dev do
  set dev_mode: true
  set include_erts: false
  set cookie: :"W?cN_`G<>ayUI&ku{<$3w7J<^nUBRBu[F[…]"
end
environment :prod do
  set include_erts: true
  set include_src: false
  set cookie: "^_`fz{dk|`w.n3Z%T,n=F>ezazFk.1ci5}[…]"
end
release :clock do
  set version: current_version(:clock)
  set applications: [
    :runtime_tools
  ]
end

Here is what the options included in the configuration file do:

• dev_mode: true/false: If true, bytecode, assets and other files are not copied to the Release folder. Instead, Distillery creates symlinks. This makes creating the Release faster and is great for testing purposes.
• include_erts: true/false: If true, Distillery includes the Erlang Runtime ERTS. This is not necessary on your development machine but recommended for making self-contained Releases.
  If you want to deploy your Release to a machine with a different operating system or processor architecture, you can specify the path to a cross-compiled version of ERTS instead of true/false.
• cookie: $STRING: Distillery can set the Erlang cookie. The auto-generated configuration includes random Erlang cookies. Don’t include a hard-coded cookie if you want to commit your configuration to a Git repository! Later in this post we’ll look into a better way to configure the magic cookie.
• include_src: true/false: This option is available for backwards compatibility with the Erlang Release tool relx. If true, Erlang source code from your project and dependencies is included in the Release. Elixir code is never included, so most of the time, you can ignore this option.
• version: $VERSION_STRING: Sets the version of the Release. You can use the function current_version/1 to extract the version from your project’s mix.exs.
• applications: [:app_name]: Here you can list additional applications that you want to include. By default, Distillery includes :runtime_tools which is part of the Erlang standard library and enables certain debugging features.

Lifecycle Scripts aka Hooks

It is possible to add shell scripts to your Release to run at certain points of your application’s lifecycle. These scripts are also referred to as hooks and can be configured in your rel/config.exs. Simply add them to an env or release section like this:

environment :prod do
  set pre_start_hook: "rel/hooks/pre_start.sh"
end

Notice that the paths are relative to your project directory and not relative to the rel directory. The most interesting hook is certainly pre_start, but there is also post_start , pre/post_configure , pre/post_stop, and pre/post_upgrade. You can currently only configure one script per hook with Distillery although Erlang would theoretically allow for more.

Umbrella Apps

Distillery is also great when working with Umbrella apps! There are really only two things you need to pay attention to:

1. Include the names of your child applications in the applications list of the Release configuration.
2. You can choose to take the Release version number from any of the child applications by using current_version(:my_child_app).

As mentioned before, it’s also possible to define more than one Release in your rel/config.exs and thus specify multiple Release profiles. This can be useful if you want to have Releases that only include certain child applications from your umbrella project.

Runtime Configuration With Releases

Configuring Elixir applications — especially when it comes to third-party libraries, can be a tricky affair. Ecto core-developer Michał recently voiced his concern about the confusing status quo.

The Problem with Mix Configuration

The first thing you need to remember is that configuration in Mix configuration files is evaluated at compile-time. This is not a problem when you run your application from source code. But when Distillery builds a Release, all configuration files are evaluated at build-time.

If you have anything like foo: System.get_env("BAR_VAR") in a Mix.Config file, you’re out of luck. foo will take on the value of the BAR_VAR environment variable from your build system. When you launch the application on another machine, the value won’t be updated.

Fortunately, there are ways around this. The easiest one is a little trick employed by Distillery:

A Simple Solution: REPLACE_OS_VARS

Distillery has a special way of injecting environment variables into Mix configuration. Simply use a string like "${BAR_VAR}" in your config file and you’re good to go. Distillery will replace it with the corresponding environment variable value when your application starts. All you need to do for this to work is setting the environment variable REPLACE_OS_VARS=true when launching your release. This makes it very easy to write configuration like this:

config :my_app,
  foo: "${BAR_VAR}"

This method is great for initializing the Erlang cookie. Use set cookie: "${ERLANG_COOKIE}" in your rel/config.exs to set the cookie from an environment variable.

REPLACE_OS_VARS doesn’t come without problems. Most importantly, this method only works with strings. If you try something like String.to_integer("${BAR}"), you’ll get an error message because the compiler evaluates the expression before Distillery gets a chance to replace the special string.

Fortunately, many libraries now support a more flexible approach to configuration.

Ecto

Ecto has introduced a new way of dynamically configuring application dependencies. Their solution: A callback! You can dynamically configure Ecto with the Ecto.Repo.init/2 callback.

This callback takes type and the config as arguments. We can safely ignore type (which is either :supervisor or :dry_run, depending on the context of the call). config is the configuration Keyword list from your Mix configuration. This is convenient because it allows you to set default values in your Mix configuration and then override them with environment variables.

The callback has to return {:ok, config} or :ignore. Here is a simple example with type conversion:

Instead of writing your own code for parsing environment variables, you might find a third-party application such as Atmo or Confex useful. I haven’t tried them yet but they seem to have some nice convenience features.

Phoenix

Creating a Release from a Phoenix application is easy.

Before you create a production release, make sure your assets are built and digested:

./node_modules/brunch/bin/brunch b -p
MIX_ENV=prod mix phx.digest

With version 1.3, Phoenix has also adapted the Ecto’s callback-style configuration method. This makes it very easy to flexibly configure Phoenix.Endpoint:

The init/2 callback for Phoenix.Endpoint behaves exactly like in Ecto.

Upgrading Releases

No (pretty) complete guide to Elixir Releases would be (pretty) complete without mentioning one of their coolest features: Hot updates.

You probably know that Erlang and Elixir are compiled to bytecode which is then executed by the BEAM virtual machine. And thanks to BEAM, we can upgrade an application without ever stopping it.

Creating an Upgrade Release

Release upgrades internally use .appup files to determine which modules should be upgraded and how. Fortunately for us, Distillery creates them automatically when we pass it the --upgrade parameter. Make sure the previous Release is still available in your /_build folder when building an upgrade. Otherwise, Distillery wouldn’t be able to determine which modules need updating.

Creating a Production Release

Only Releases that include ERTS can use hot upgrading. So, let’s start by creating a production Release of our clock application:

#In Terminal 1
MIX_ENV=prod mix deps.get
MIX_ENV=prod mix release

Now open up a second terminal window and launch the Release:

#In Terminal 2
cd distillery-clock-demo
_build/prod/rel/clock/bin/clock foreground.

Again, you should see the current time printed out every second.

Creating an Upgrade Release

Let’s create a new version of clock. Since it prints out its current version number together with the current time, all you need to do to get a different output is change the version in mix.exs. Currently it’s set to 0.1.0. Let’s change it to 0.2.0.

Then, create an upgrade Release by passing the --upgrade flag to mix.release.

#In Terminal 1
MIX_ENV=prod mix release --upgrade

We’ll Do It Live!

After the Release upgrade has been created, the Clock application in the second terminal hasn’t changed. Yet. Still in the first terminal, run the following command:

#In Terminal 1
_build/prod/rel/clock/bin/clock upgrade 0.2.0

In the second terminal, the Clock output will now change and start to also print out the current date. How neat!

Updating an application wile it’s running!

Are You a Master of Elixir Releases Yet?

I hope that after reading this guide, you now have a better understanding of Elixir Releases . I certainly do after writing it!

If you think I missed an aspect of Elixir Releases that should be part of this (pretty) complete guide, please let me know and I will try to add it!

This article is part of an ongoing series about developing and deploying Elixir applications to production. Upcoming articles will cover deployment strategies and building Docker images.

In this series I am sharing my experience from creating DBLSQD, a release and update server written in Elixir. Check it out, there is a 60-day no-strings-attached free demo:

Publish & Distribute Software — DBLSQD