Continuous Delivery for Elixir (Part 4 — Use a release, troubles cease)
Where we’re headed
In the previous section, we set up our project with a dynamic configuration file. In this section, we’ll create an OTP release, which contains our application, all our hex.pm dependencies, our Elixir language dependency, and the BEAM. This release will also examine our dynamic configuration file and incorporate the values upon launch.
Catch and release
If we review our pipeline from part 1, we see that I said the release would be built as part of Jenkins pipeline. That’s true, but not quite yet. First we’ll want to prove out the process on a development machine before migrating it over to Jenkins. We’ll have faster iteration this way while we work on refining the release and the packaging. In a later section, I’ll cover moving this portion of the pipeline over to Jenkins.
The tool we’ll use for generating our OTP release is exrm, which is in turn, built upon relx. Exrm integrates very well with our external .conf via the conform_exrm plugin. Not surprising since Paul Schoenfelder wrote conform, exrm, and conform_exrm (amongst other great libraries). If you haven’t thanked Paul recently, do it now. I’ll wait. 💜
Here’s what the deps and application sections of our mix.exs look like now.
defp deps do
[
{:exml, github: "paulgray/exml", override: true},
{:hedwig_hipchat, "~> 0.9"},
{:romeo, "~> 0.4.0", override: true},
{:conform, github: "bitwalker/conform", override: true},
{:exrm, "~> 1.0"},
{:conform_exrm, "~> 1.0"},
]
enddef application do
[ applications: [
:logger,
:exml, :romeo, :hedwig, :hedwig_hipchat,
:conform, :exrm, :conform_exrm,
],
mod: {Alfred, []}
]
end
We can verify proper loading of the conform_exrm plugin.
$ mix release.plugins
protocol.consolidation # Performs protocol consolidation
appup # Generates a .appup for each dependency
conform # Generates a .conf for your releaseGreat, we’ll generate a .conf for our release! I know, isn’t that what we did in the last section? Yes, kind of. We did generate alfred.conf, but without the conform_exrm plugin, even if we managed to shoehorn alfred.conf into the OTP release, it wouldn’t be consulted. While conform_exrm may seem extraneous at first, it’s a vast improvement to lost hours (pre-1.0) trying to figure out that my .confs weren’t used because I hadn’t force recompiled either exrm or conform (I don’t remember which) after compilation of the other. The conform_exrm plugin makes this repeatable and predictable, and that’s what we care about.
We’ll talk a bit more about the appup plugin later. For now, I’ll simply mention that appup will enable you to do a hot upgrade of your running system.
A handful of other promising exrm plugins exist, including those for building RedHat and Debian packages. We won’t use them in this series, but they’re worth investigating to see if they meet your needs.
Let’s build a production release and look at what is provided.
$ MIX_ENV=prod mix do compile, release
Building release with MIX_ENV=prod.
==> The release for alfred-0.0.1 is ready!
==> You can boot a console running your release with `$ rel/alfred/bin/alfred console`Inside rel/alfred, we have scripts in bin that allow us to start our application, an entire Erlang runtime system, our application and all its dependencies, and release-specific information (like which versions of the dependencies to use, etc).
$ tree -L 2 rel/alfred
rel/alfred
├── bin
│ ├── alfred
│ ├── alfred.bat
│ ├── install_upgrade.escript
│ ├── nodetool
│ └── start_clean.boot
├── erts-7.3
│ ├── bin
│ ├── include
│ ├── lib
│ └── src
├── lib
│ ├── alfred-0.0.1
│ ├── asn1-4.0.2
│ ├── bbmustache-1.0.4
│ ├── cf-0.2.1
│ ├── compiler-6.0.3
│ ├── conform-2.0.0
│ ├── conform_exrm-1.0.0
│ ├── connection-1.0.2
│ ├── crypto-3.6.3
│ ├── elixir-1.2.5
│ ├── erlware_commons-0.19.0
│ ├── exml-2.2.1-7-gd04d0df
│ ├── exrm-1.0.5
│ ├── getopt-0.8.2
│ ├── gproc-0.5.0
│ ├── hedwig-1.0.0-rc3
│ ├── hedwig_hipchat-0.9.4
│ ├── iex-1.2.5
│ ├── kernel-4.2
│ ├── logger-1.2.5
│ ├── neotoma-1.7.3
│ ├── providers-1.6.0
│ ├── public_key-1.1.1
│ ├── relx-3.19.0
│ ├── romeo-0.4.0
│ ├── sasl-2.7
│ ├── ssl-7.3
│ └── stdlib-2.8
└── releases
├── 0.0.1
├── RELEASES
└── start_erl.dataIt’s worth noting that the Erlang runtime system (ERTS) that’s included in the release is one from your development machine. So unless you’re developing on a Debian-based OS, the release will not be completely compatible with our target Debian 8 production host. We could cross-compile our release, either with no ERTS or with one compatible with our target system, but that’s beyond the scope of this series. For this series, our eventual Jenkins worker will use an ERTS that is compatible with our production target.
Examining the sys.config from our release, we see the Erlang Term Format version of our conform.effective from the previous section.
$ cat rel/alfred/releases/0.0.1/sys.config
[{sasl,[{errlog_type,error}]},
{alfred,
[{'Elixir.Alfred.Robot',
[{adapter,'Elixir.Hedwig.Adapters.HipChat'},
{name,<<"Alfred">>},
{aka,<<"!">>},
{jid,<<"587458_3985539@chat.hipchat.com">>},
{password,<<"not_my_real_password">>},
{rooms,
[{<<"587458_testing@conf.hipchat.com">>,[]},
{<<"587458_elixircontinuousdelivery@conf.hipchat.com">>,[]}]},
{responders,
[{'Elixir.Hedwig.Responders.Help',[]},
{'Elixir.Hedwig.Responders.GreatSuccess',[]},
{'Elixir.Hedwig.Responders.ShipIt',[]}]}]}]}].We know from the “mix release” output that we can run “rel/alfred/bin/alfred console” to start the application, but what else can we do?
$ rel/alfred/bin/alfred
Usage: alfred {start|start_boot <file>|foreground|stop|restart|reboot|ping|rpc <m> <f> [<a>]|console|console_clean|console_boot <file>|attach|remote_console|upgrade|escript|command <m> <f> <args>}Cool, we can start, stop, connect a remote console.
As “mix release” suggested earlier, let’s run “rel/alfred/bin/alfred console.”
$ rel/alfred/bin/alfred console
copying /Users/jeff/cdfe/alfred/rel/alfred/releases/0.0.1/alfred.conf to /Users/jeff/cdfe/alfred/rel/alfred/running-config/alfred.conf ...
using /Users/jeff/cdfe/alfred/rel/alfred/running-config/alfred.conf to populate "/Users/jeff/cdfe/alfred/rel/alfred/running-config".
...
Erlang/OTP 18 [erts-7.3] [source] [64-bit] [smp:8:8] [async-threads:10] [hipe] [kernel-poll:false] [dtrace]
Interactive Elixir (1.2.5) - press Ctrl+C to exit (type h() ENTER for help)
iex(alfred@jweiss)1>
...Wow, we see alfred.conf copied from our release to the running configuration (because we may have several prior releases installed) and then that running configuration is used to generate our sys.config. We’ve started an IEx session, and we’ve picked up the hostname for our Distributed Erlang session.
Alongside the sys.config in our rel/alfred/releases/0.0.1 directory, we also have a tarball of the entire release: rel/alfred/releases/0.0.1/alfred.tar.gz. This tarball is the artifact that will transition between our “release” and our “package.”
Let’s take a moment to reflect upon how far we’ve come, because even if we stopped here and manually deployed to tarballs to our production hosts, without system packages, and without automated configuration management, we have a dependency-free tarball, containing a start/stop script, and a working key=value configuration file. We really are beginning to look just like another application to our Ops team! A happy Ops team is, well, mythical, but we’ve drastically made their lives less unhappy (when working with our application, at least).
What’s next?
We have working tarball release with mostly-LSB compatible init script. In the next section, we will add a small wrapper with the last bit needed for LSB compliance, look for our configuration in a standard location, and build a system package. Again, we’ll do this locally before moving it to Jenkins.
