How to automate the deployment of your Elixir project with CircleCI
This is the second article of a 2-part series, where we explain in full how to automate your Elixir project’s deployment. The technologies used are AWS, to host your project, and CircleCI, to automate the testing and deployment of the application.
Part 1 — How to automate your Elixir project’s deployment into AWS
Part 2 — How to automate the deployment of your Elixir project with CircleCI
Continuous… Integration? Delivery? Deployment?
Before we start to explain any of the steps we are going to take, there is a need to define exactly what we want to achieve with this tutorial, so let’s set the stage.
- You have an Elixir application on Github or any other Git platform, with branches for the features, but more importantly, branches dedicated to the the staging and production environments.
- You have the necessary infrastructure which you want to deploy the code to — check the previous article to know how to configure AWS. This should make your application accessible to testers (staging) and to customers (production).
At Coletiv, we try to automate the workflow of our projects up to a certain point. We like to automate the tests, as well as the deployment to the staging environment — Continuous Delivery — but we prefer to do the production deployment manually.
This tutorial describes the deployment in this optic, but you can as easily change the steps to adapt your needs. You can achieve Continuous Deployment by taking the steps that automate the staging environment and applying them to the production environment.
A) Preparing Your App for Deployment!
So, where to start? You will be executing these steps on your own machine, where you have a local version of some feature branch of your project that you want to deploy. These are crucial configurations that only have to be done the first time.
1. Adding New Elixir Dependencies for Release
There are 2 new depedencies that make deploying elixir/phoenix projects a lot easier. These are “edeliver” and “distillery”. While distillery produces a release out of your mix project, edeliver deploys the release to your specified environment.
First, go to the mix.exs file and add them to the dependencies. Also add edeliver to the applications, just as follows:
def application, do: [
applications: [
...
# Add edeliver to the END of the list
:edeliver
]
]defp deps do
[
...
{:edeliver, "~> 1.6.0"},
{:distillery, "~> 2.0", warn_missing: false},
]
end
2. Configuring Distillery
Next, you will be writing some additional configurations, for these dependencies to work correctly with the infrastructure you prepared.
Distillery has a nice command that prepares a new project for releases. Try running it:
mix release.initIt should create a folder named rel in the project root, with a config.exs file inside of it. Navigate to this file, find the line that sets the applications and replace it with the following:
applications: [
:your_app,
:edeliver
]3. Configuring Edeliver
Create a folder named .deliver in your project root, if it does not yet exist. Inside of it, create a file named config (no extension). If you followed the previous tutorial, about AWS, the contents you have to put into your config file should be something like this:
Do not forget to actually replace “your_app_name” and “your_aws_ec2_staging_machine_ip” with the actual values (leave the quote marks untouched).
This is the file that configures WHERE your code will be deployed to, anytime we merge a new feature to the staging or production environments. We usually leave the staging IP by default, so that CircleCI will use it for the deployment to the staging machine.
For now, we are automating staging only, so you can leave the file with the staging machine’s IP. If you wanted to actually do a direct deployment for production, you would need to make the following changes:
- infoChange the IP to your production machine’s IP instead.
- Replace STAGING_HOSTS with PRODUCTION_HOSTS and STAGING_USER with PRODUCTION_USER
The next deployment to be triggered, either manually or not, would use that IP.
Tip: You might want to add the folder .deliver/releases to git ignore.
echo ".deliver/releases/" >> .gitignore4. The Deployment Scripts
We created 2 deployment scripts: deploy.sh for staging and prod_deploy.sh for production. Go to the project root, create a folder named scripts and inside of it, create the files for these scripts.
deploy.sh
prod_deploy.sh
These scripts have all the necessary steps to build and deploy the project according to the previous configurations.
5. Creating the Configuration for Secret Credentials — prod.secret.exs
We might already have everything we need to manually deploy a project, but one important detail is missing. Some of the configurations for your application, such as the database credentials, should not be stored in the code you commit …
Instead, you should store all the sensible data in a separate file, which will be manually placed on the staging/prod machines. The configurations written in this file have the same syntax as a normal config file from your project. For instance, if you have a database you want to access with secret credentials, you should include the configuration for it in this new file:
use Mix.Config
# Configure your database
config :app_name, AppName.Repo,
database: "database_name",
username: "username",
password: "my_password",
hostname: "host_name"Let’s create this file and call it prod.secret.exs. SSH into your staging/prod machines, navigate to the folder /opt/you_app_name and create the file there, or move it from your own local machine.
Feel free to add any other dependency configurations from your project to the file, especially if they include secret credentials or keys!
5. Endpoint Configurations on prod.secret.exs
Next, the prod.secret.exs file needs some extra configurations, if you want to be able to access the projects endpoints remotely.
# Configures the endpoint
config :<your_app_name>, <YourAppNameWeb>.Endpoint,
secret_key_base: "<the_key>",
http: [port: <your_nginx_port>],
load_from_system_env: falseconfig :your_app_name, <YourAppNameWeb>.Endpoint,
server: true,
check_origin: false,
debug_errors: true# CORS
config :cors_plug,
origin: [
"http://localhost:3000",
"http://staging.your_app.app"],
max_age: 86400,
methods: ["GET", "POST", "PUT", "DELETE", "PATCH"]
Replace <your_app_name> by the name of your app and <YourAppNameWeb> by the name of the Web module of your app. Next, replace <your_nginx_port> by the port you might have configured in the AWS tutorial. By default, it should be 4000, so you can try that.
Now, run the command:
mix phx.gen.secretReplace <the_key> with the result from this command.
Finally, you only need to configure the CORS dependency. If your project has API routes, CORS will enable client applications to call this API.
Therefore, replace the origin links with whatever domains are going to be calling/accessing your API. Additionaly, add the CORS dependency locally, on your project’s mix.exs file:
# CORS{:cors_plug, “~> 1.5”}
Do not forget to adapt the contents of the file to the environment of the machine. For instance, if the staging and production machines use different databases, then the prod.secret.exs file in both machines should differ in that configuration!
Now, you only need to import the secret file to your project. Go to your local config/prod.exs file and add the following line to the END of the file:
import_config "prod.secret.exs"Voilá! You now have your secret credentials safely stored in the machines and your project knows how to access them!
7. Running the First Deployment to Staging
The first ever deployment to staging is manual, in order to populate the machines with the project’s code.
In your local machine, open the console, navigate to the project folder, checkout to the develop branch of your project and run the following:
export CI_BRANCH=develop
export CI_PULL_REQUEST=false
scripts/deploy.shThis will prompt a manual deployment to the machine whose IP is on the .deliver/config file, so make sure that you have the correct staging IP and project details in that file. Now, just wait for the deployment to finish…
Congratulations! The first deployment was finished.
A few very important details to take into account before any deployment:
Tip 1: Always change the .deliver/config file to reflect the environment you are deploying to (e.g.: change the IP to the production machine IP)
Tip 2: Change the project version in mix.exs so you can later check if the deployment was successful
Tip 3: Update prod.secrets.exs if necessary
B) Setting up CircleCI for Continuous Delivery!
It is now time to automate the testing and deployment processes, if we want to achieve Continuous Delivery.
The objective is that, whenever a Pull Request from a feature to the staging branch is made, the code is tested and then deployed when the Pull Request is accepted. For this end, we will be using CircleCI.
1. Create a Context for your Project
In your project, you probably use some environment variables. Some of them are more secret, some are not as much. In your local machine you might place them in a .env file (which normally is is local and uncommited). In the staging/prod machines, you place these in the prod.secret.exs file.
But what about CircleCI? It does not have access to your machine’s prod.secret.exs nor your local .env file! This means you need to provide it with the variables yourself, and this is what Contexts are for.
Navigate to the Settings tab, choose the Contexts option and then Create Context.
Access the context you just created and add any environment variables CircleCI might need to run your project, like database passwords and other sensitive application configurations (like the ones on the prod.secrets.exs file).
You may also add 2 important variables that were previously mentioned.
Name: CI_BRANCH Value: develop
Name: CI_PULL_REQUEST Value: false2. Adding the Project to CircleCI
Login to CircleCI with the Github account you have access to the project with. On the left menu, navigate to the “Add Projects” tab and choose your project from the list.
This will open a page with a set of instructions. For Elixir, the instructions are as follows:
2.1 On your project folder, create a folder named .circleci and inside of it, a file named config.yml.
2.2 Copy the following contents to the file you just created:
2.3 Update the contents of the file to reflect your own projects details.
For instance, you may change lines 83 and 85 to the name of the context that you created in the previous step.
In lines 7 to 12, check the versions of the software you are using and change them accordingly.
2.4 Push this change to GitHub
2.5 Return to the page with the set of instructions in CircleCI and press the “Start Building” button.
3. Giving CircleCI the permissions to access the Staging machine
The previous steps might all be correct, but without the private key to the staging machine, CircleCI still can’t finish the deployment.
Navigate to the project you just added, enter the settings and on the left bar, find the SSH Permissions. Next, press Add SSH Key, and you will be prompted with the following window.
Simply add as hostname the IP to the staging machine and copy the private key (which you should have, if you followed the AWS tutorial). And this is it!
You’re done with setting up CircleCI! Your code should now run the tests and deployment automatically, as expected.
C) Creating a Release and Deploying to Production
The following steps describe how to manually create a release on Github and how to deploy a release to the production servers.
- Create a Pull Request from your Staging branch to Master, accept and merge it.
2. Create a release on Github based on your master branch — remember the version tag, you will use it later.
3. Get the code on your local machine, compile and test it.
checkout master
git pull
mix deps.get
mix compile
mix test4. Edit the .deliver/config file. First, change all the IPs to the IP of the production machine. Then, replace STAGING_HOSTS with PRODUCTION_HOSTS and STAGING_USER with PRODUCTION_USER.
5. On the project folder, run the production deployment script.
scripts/prod_deploy.sh6. When you are prompted with the question “Release or update?”, choose “Release”.
7. When asked about the release version, choose the one corresponding to the latest version on your mix.exs file. This should also be the version that you used on the tag of your github release.
8. Wait for the deployment to finish
9. You are done!
This wraps up our tutorial, as your deployment to production was probably successful at this stage! We leave you with some extra tips below.
Tip 1: Whenever doing future features, do not forget to check that the .deliver/config file has the IP for staging correctly configured. Also check that the file has a STAGING_HOSTS and STAGING_USER field when deploying to staging and PRODUCTION_HOSTS and PRODUCTION_USER when deploying to production!
Tip 2: To check if the deployment was successful, maybe consider adding an /api/info route to your project, which prints the version of the application.
Tip 3: You may use all of the previously provided scripts and try to alter them to fit your needs or even automate your deployment to production
Thank you for reading!
I hope this article helps you and your team to successfuly automate the deployment of your projects, saving you a lot of time.
Thank you so much for reading and if you enjoyed this article make sure to hit that 👏👏 button. It means a lot to us! Also don’t forget to follow Coletiv on Medium, Twitter, and LinkedIn as we keep posting more and more interesting articles on multiple technologies.
In case you don’t know, Coletiv is a software development studio from Porto specialised in Elixir, iOS, and Android app development. But we do all kinds of stuff. We take care of UX/UI design, web development, and even security for you.