Deploying an Elixir application with edeliver
Previously, we talked about distillery and how to create a release and an upgrade
But that was the first step. Today we are going to talk about edeliver a tool that will help us to deploy our application. But what is edeliver?. As it says in its github page
edeliver is based on deliver and enables you to build and deploy Elixir and Erlang applications and perform hot-code upgrades.
The erlang releases are built on a remote host that is similar to the production machines. After being built, the release can then be deployed to one or more production machines.
Once built, the release contains the full erts (erlang runtime system), all dependencies (erlang or elixir applications), the Elixir runtime, native port drivers, and your erlang/elixir application(s) in a standalone embedded node.
But, what does that means?
Well, it means that we are going to need another machine besides our development machine and our deployment machine. But, fear you must not my young padawan, Ansible, Vagrant and virtualbox our allies are. With those tools we will recreate our deployment enviroment.
By the way, our deployment target will be a CentOS machine.
Warming up (Requirement tools)
First things first, be sure to download and install Ansible, Vagrant and virtualbox in your machine.
With those tools already installed, clone the following repository ansible_elixir move to the directory created and run vagrant up. Once is finished, open your virtualbox and you will see the following:
On the left side you will see two virtual machines created by vagrant and configured by ansible
I’m not going to explain how to configure ansible. If you want to know more about it, you can check this amaizing book or its documentation
Now we need to verify that our build machine is running elixir and erlang. And test knows nothing about them. For that, we are going to use an Ansible feature called Ad-Hoc Command.
An Ad-Hoc Command is nothing more than a command writen in our shell but executed into one or more virtual machines. Of course you can always connect to virtual machines via ssh but, if we do it that way, there would be no need for Ansible.
❯ ansible -i provisioning/inventory local -b -a "which erl"
192.168.60.8 | FAILED | rc=1 >>
which: no erl in (/sbin:/bin:/usr/sbin:/usr/bin)
192.168.60.7 | SUCCESS | rc=0 >>
/bin/erl
❯ ansible -i provisioning/inventory local -b -a "which elixir"
192.168.60.8 | FAILED | rc=1 >>
which: no elixir in (/sbin:/bin:/usr/sbin:/usr/bin)
192.168.60.7 | SUCCESS | rc=0 >>
/bin/elixirAs we can see, the virtual machine with ip address 192.168.60.8 knows nothing about erlang or elixir. Meanwhile, 192.168.60.7 knows everything about them.
With that in place we will prepare our application for deploy.
Early game (Preparing our deploy)
The first thing you will need to do is fork the next repo issue_timer and cloning it. At this moment, the only thing that this application is doing is:
- Conect with github
- Show the user’s information and his repos
In order to follow this, you will need to create an application in github.
Once you clone the repo into your machine, change it to the tag before_edeliver
❯ git checkout before_edeliver -b before_edeliver
Switched to a new branch 'before_edeliver'Now we need to update our mix.exs to includeedeliver and distillery as dependencies.
defp deps do
[
{:phoenix, "~> 1.3.0"},
{:phoenix_pubsub, "~> 1.0"},
{:phoenix_ecto, "~> 3.2"},
{:postgrex, ">= 0.0.0"},
{:phoenix_html, "~> 2.10"},
{:phoenix_live_reload, "~> 1.0", only: :dev},
{:gettext, "~> 0.11"},
{:cowboy, "~> 1.0"},
{:ueberauth_github, "~> 0.4"},
{:tentacat, "~> 0.5"},
{:edeliver, "~> 1.4.4"},
{:distillery, "~> 1.4", runtime: false}
]
endAfter that, we fetch dependencies and compile project.
❯ mix do deps.get, compile
Running dependency resolution...
Dependency resolution completed:
certifi 2.0.0
connection 1.0.4
cowboy 1.1.2
cowlib 1.0.2
db_connection 1.1.2
decimal 1.4.0
distillery 1.4.1
ecto 2.1.6
edeliver 1.4.4
exjsx 3.2.1
fs 0.9.2
gettext 0.13.1
hackney 1.9.0
httpoison 0.13.0
idna 5.1.0
jsx 2.8.2
metrics 1.0.1
mime 1.1.0
mimerl 1.0.2
oauth2 0.9.1
phoenix 1.3.0
phoenix_ecto 3.2.3
phoenix_html 2.10.3
phoenix_live_reload 1.0.8
phoenix_pubsub 1.0.2
plug 1.4.3
poison 3.1.0
poolboy 1.5.1
postgrex 0.13.3
ranch 1.3.2
ssl_verify_fun 1.1.1
tentacat 0.6.2
ueberauth 0.4.0
ueberauth_github 0.5.0
unicode_util_compat 0.3.1
* Getting edeliver (Hex package)
Checking package (https://repo.hex.pm/tarballs/edeliver-1.4.4.tar)
Using locally cached package
* Getting distillery (Hex package)
Checking package (https://repo.hex.pm/tarballs/distillery-1.4.1.tar)
Using locally cached package
==> distillery
Compiling 19 files (.ex)
Generated distillery app
==> edeliver
Compiling 32 files (.ex)
Generated edeliver app
==> issue_timer
Compiling 20 files (.ex)
Generated issue_timer appOnce is finished, we will create a file in .deliver/config as follows:
Change APP="" for APP="issue_timer". But, what does that file contains? Well, that file contains the information about our build server (where the application compiles) and our deployments servers.
This file contains hooks that you can use for customizing your deployment, in case that you need something special for your application. If you need more information about running additional tasks you can click here. And for information about configuration check this
In case you did not realize, BUILD_HOST and STAGING_HOST, both have the ip that we setted before with ansible. That means that we are going to compile and deploy our application in VirtualBox.
Before we go further, we need to update our prod.secret.exs file as follows:
use Mix.Config
config :issue_timer, IssueTimer.Repo,
adapter: Ecto.Adapters.Postgres,
username: System.get_env("DATABASE_USERNAME"),
password: System.get_env("DATABASE_PASSWORD"),
database: System.get_env("DATABASE_NAME"),
hostname: System.get_env("DATABASE_HOST"),
port: System.get_env("DATABASE_PORT"),
pool_size: 10
config :ueberauth, Ueberauth.Strategy.Github.OAuth,
client_id: System.get_env("GITHUB_CLIENT_ID"),
client_secret: System.get_env("GITHUB_CLIENT_SECRET")Also we need to update our prod.exs in order to tell phoenix that start all endpoints and to change host and port to take his value from ths system:
use Mix.Config
config :issue_timer, IssueTimerWeb.Endpoint,
load_from_system_env: true,
url: [host: {:system, "HOST"}, port: {:system, "PORT"}],
cache_static_manifest: "priv/static/cache_manifest.json"
config :logger, level: :info
config :phoenix, :serve_endpoints, true
import_config "prod.secret.exs"And we need to create config/profile file. This file should export the variables that our prod.secret.exs needs for compilation:
export DATABASE_USERNAME=postgres
export DATABASE_PASSWORD=postgres
export DATABASE_NAME=issue_timer_prod# local machine
export DATABASE_HOST=192.168.15.5
export DATABASE_PORT=5432export GITHUB_CLIENT_ID=
export GITHUB_CLIENT_SECRET=
This file should be ignored in
.gitignore. And for obvious reasons, I’m not going to put mygithubcredentials here.
And for last but not least, we need to initialize our project with distillery
❯ mix release.init
Compiling 20 files (.ex)
Generated issue_timer app
An example config file has been placed in rel/config.exs, review it,
make edits as needed/desired, and then run `mix release` to build the releaseMid game (Building our application in build-host)
Before build our binary, we will create a tag called with_edeliver that’s for telling to edeliver which changes we need in this release.
❯ git tag -a with_edeliver -m "Setup edeliver and distillery"
❯ git push origin --tagsAfter that, we can continue with our release:
❯ mix edeliver build release --tag=with_edeliver
BUILDING RELEASE OF ISSUE_TIMER APP ON BUILD HOST
-----> Authorizing hosts
-----> Ensuring hosts are ready to accept git pushes
-----> Pushing new commits with git to: vagrant@192.168.60.7
-----> Resetting remote hosts to 3ab2b9f78b4e21be590ebb6a2d40789fea652b9a
-----> Cleaning generated files from last build
-----> Copying '/Users/makingdevs/Documents/issue_timer/config/prod.secret.exs' file to build host
-----> Fetching / Updating dependencies
-----> Copying '/Users/makingdevs/Documents/issue_timer/config/profile' file to build host
-----> Preparing assets with: brunch build and phoenix.digest
-----> Compiling sources
-----> Generating release
-----> Copying release 0.0.1 to local release store
-----> Copying issue_timer.tar.gz to release store
RELEASE BUILD OF ISSUE_TIMER WAS SUCCESSFUL!By the way, don’t forget to export WORKSPACE environment variable with the value of the parent directory. For example if your application is in
~/Documents/issue_timeryour export should look likeexport WORKSPACE=~/Documents
And that’s all? NO!
Late game (Deploying into test-host)
Now we need to deploy our binary to staging environment. And we do that in the following way.
❯ mix edeliver deploy release to staging --version=0.0.1
DEPLOYING RELEASE OF ISSUE_TIMER APP TO STAGING HOSTS
-----> Authorizing hosts
-----> Uploading archive of release 0.0.1 from local release store
-----> Extracting archive issue_timer_0.0.1.tar.gz
DEPLOYED RELEASE TO STAGING!This ain’t over, we need to start our system and run migrations.
❯ mix edeliver start staging
EDELIVER ISSUE_TIMER WITH START COMMAND
-----> starting staging servers
staging node:
user : vagrant
host : 192.168.60.8
path : /home/vagrant/releases
response:
START DONE!You need to prepare your postgres to allow connections from
192.168.60.8. If you are like me and don’t know how to do that, you can check this post
First we need to create the database. For that, login into postgres and run create database issue_timer_prod. Then we need to be sure that we have pending migrations in our systems. We can do that by executing the following:
❯ mix edeliver show migrations on staging
EDELIVER ISSUE_TIMER WITH MIGRATIONS COMMAND
-----> migrationsing staging servers
staging node:
user : vagrant
host : 192.168.60.8
path : /home/vagrant/releases
response: ==> pending: create_users (20170807195932)
ok
MIGRATIONS DONE!In the response we can see that we have one pending migration called pending: create_users (20170807195932). You can run this migration with the following command:
❯ mix edeliver migrate staging
EDELIVER ISSUE_TIMER WITH MIGRATE COMMAND
-----> migrateing staging servers
staging node:
user : vagrant
host : 192.168.60.8
path : /home/vagrant/releases
response: [20170807195932]
MIGRATE DONE!And that’s all for our migrations! You can check one more time if you still have pending migrations. And you should see an ok reponse, where the pending migrations was.
Now login into your github account and change your authorization callback of your application to the new url.
Open your browser and hit http://192.168.60.8:4000, login into the system and you will see your profile page.
Now we can be happy because that’s all for our deployment.
Game analisis (Conclusions)
After seeing all of this we can be sure on one thing, is a lot of WORK!. But, hey! what kind of deployment is not a hard work?. The only thing different here is that with this setup we can run another deploy easily with mix edeliver build release and for making and upgrade it will be a piece of cake. But that we will see in another post.
By now, you should know how distillery and edeliver is interacting, how you can Run additional build tasks and how to configure your edeliver.
Also you already have a glimpse on how Ansible can help you on further developments and for building environments.
If you want to know more about what can you do with edeliver check his help into the command line with mix edeliver --help
I hope you can find interesting and helpful. If you have any comments or any doubts please let me know.
And as with the previous post, in case you have troubles, here is the main repository to download this example. And if you want to review step by step on how the configuration was, please check the releases page
See you next time. Good luck and have fun!
