How to setup a continuous deployment (CD) pipeline with Travis continuous integration (CI) and AWS:
As detailed here (https://medium.com/full-taxx/how-to-setup-travis-ci-for-a-rails-application-78a453963300), our 1.14 week old rails project has already been setup with continuous integration. Now, we are ready for the next fancy addition: Continuous Deployment.
In my understanding as a barely-12-week-old developer, continuous deployment is essentially automatic deployment (i.e. ‘putting on the internet’) of software. Continuous integration (CI) integrates testing with version control by monitoring our github repository for changes, making testing a continuous (e.g. seamless) part of the build. In a similar way, continuous deployment involves monitoring the main, master branch of our respository for changes, and if the CI tests pass, automatically deploying the latest version of our master branch into the digital yonder.
This is fairly minimal in terms of effort-saving, as it doesn’t take many keystrokes to just deploy the master yourself. But it does make a philosophical difference. Anything you put in the master will be automatically put on the public-facing app. This means you definitely don’t put half-implemented, badly-tested or flat-out broken stuff in the master branch, because then it will be shoved into the app, and then you will have broken production and be subject to the scary-sounding ‘git blame’ procedure. The master branch becomes, as it should be, synonymous with the final product.
We setup our deployment environment on Amazon Web Services (AWS), a snazzy cloud-based platform. It’s more difficult to use than the other choice we had (Heroku), which gives it a certain coolness factor. It does a whole bunch of other, big-guys stuff like autoscaling that Heroku doesn’t do straight out of the box, which will obviously be relevant when we scale our facebook clone to upwards of 5 users. But mainly we used AWS because it’s a professional tool that we hadn’t used yet.
Continuous deployment on AWS #1: Discontinuous deployment AKA just regular old deployment
Firstly, you need to actually be able to deploy in the first stage. To deploy to AWS, you need an account (tutorial here: https://aws.amazon.com/premiumsupport/knowledge-center/create-and-activate-aws-account/). Most of the following is taken from this tutorial, which is part of the AWS docs (https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/create_deploy_Ruby_rails.html). Basically the procedure is this, assuming you have a rails already setup:
1: Installing the acronyms
Install the Elastic Beanstalk (EB) command-line interface (CLI). The EB CLI is a python based program, so it you’re running MacOS you should be fine (python is very considerately already included!). If you’re running windows I’m not vouching for anything in this tutorial, sorry — you might be on your own a bit…
Anyways, to install the EB CLI, run the following in bash:sudo pip install awsebcli
.The EB CLI will allow you to configure your AWS deployment environment (which you can think of as the server that will store and run your app on the internet) from the comfort of your own command line.
2: Configuring your deployment environment
Configure the EB CLI by running eb init
in the root folder of your app. Console prompts will appear to guide you through setting up your deployment environment.
You will then see several prompts appear in bash, asking you a bunch of stuff. Most of it is self-explanatory, but the prompts asking for your AWS access key ID and secret key involve some digging around in the AWS website.
To get these, log into your AWS account, click on your name (in the top right corner), then select my security credentials
from the drop-down menu. Click on continue to my security credentials
in the pop-up, then select the access keys
menu item. Now, select create new access key
, and copy/paste the access key ID and the access key itself into the console.
The final thing that might throw you is the set up ssh?
option. I would just leave this on n
, I don’t think you need it for the sort of admin tasks we’ll be doing.
eb init
also sets up some stuff in .gitignore that will need to be commited and pushed to github, which you can do with git commit -am "updated .gitignore".
3: Setting up your AWS application and AWS instance, deploying!
The next stage requires that we create a the environment in which we will deploy our app (this is configured by the eb cli, but still needs to be actually created in the AWS console). Go to the AWS Elastic Beanstalk console and select create new application
from the top right menu. Fill in the application name with to have the same name as your rails project.
You will now be piped to a page which says ‘No environments currently exist for this application’. Well done! Now you have to build some environments within your application (i.e. the things that can actually run your app). Thankfully the stuff you setup in the EB CLI will do this for you!
To setup an enviroment from bash, use eb create your-env-name-here
, with your-env-name-here replaced with whatever you want to call the env. This will create your enviroment on AWS and then (attempt to) deploy it!
4. It’s broken (database setup!)
If your app (like ours) involves a database, this initial deployment will fail, because the environment created by eb create
didn’t include a database!
It’s actually pretty easy to set one up though. In your AWS console on the page for your application, a new enviroment will have appeared. It will probably be red. Click on that, then click on configure
from the left-hand menu, and go to the database
box in the following page and click on modify
. The options are fairly straightforward, just make sure the type (mySQL, postgres etc) matches what you have specified in your rails app. Click save
and then in the next page click apply
.
If you then return to the dashboard for your enviroment, you can see that it’s thinking long and hard about setting up the RDS database you just specified. This will take a few minutes, then it will attempt to deploy the app again. More than likely, you’ll be confronted with…
5. It’s still broken (database passwords!)
More than likely your app still won’t work. This time it’s probably because you didn’t specify the password and username for your new AWS database in the database.yml file in your rails app. Thankfully this is pretty easy to do, just alter the production
bit of your database.yml file so that it reads like this:
production:
<<: *default
adapter: postgresql
encoding: unicode
database: <%= ENV['RDS_DB_NAME'] %>
username: <%= ENV['RDS_USERNAME'] %>
password: <%= ENV['RDS_PASSWORD'] %>
host: <%= ENV['RDS_HOSTNAME'] %>
port: <%= ENV['RDS_PORT'] %>
The ‘adapter’ and ‘encoding’ parts may be different in yours, but the ‘env’ statements should be the same. They tell the app to look in the AWS environment ENV variables for its database details.
Add those lines to the yml file, save, commit, then redeploy your app from the command line using eb deploy
. You should now see your enviroment change from red to green in your AWS console!
To visit the webapp you’ve just built (woo!), click on the URL shown in small text just after the All Applications > yourApp > yourEnv
bit at the top of the page. BUT!…
6. It’s STILL broken (secret key base stuff)
You now have spent hours of your life creating a page that says ‘502 Bad Gateway’. Brill! (NB If you now have a complete, working webpage congratulations, you worked some stuff out that we didn’t, you can stop reading this blog). If you read the error logs (select logs
from the menu on the left, then request logs, last 100 lines
from the drop-down, the download
to make them appear on another page), you should see some references to a missingsecret key base
.
This secret key base is found in the development
part of the secrets.yml
file in the app/config
folder of your rails app. This key base needs to be added as an environmental variable in your AWS env if your app is to work.
To add it, your need to run the following on base, replaced the your_env and your_key variables with the name of your environment and the value of the secret key base in your secrets.yml
file:
eb use your_env
eb setenv SECRET_KEY_BASE=your_key
Finally, you should have the following in production
part of the secrets.yml
file, to tell the app to look in the ENV for what it needs:
production:
secret_key_base: <%= ENV["SECRET_KEY_BASE"] %>
Now, save, add, commit, then deploy using eb deploy
7. It works!
HOPEFULLY. If not, you have passed beyond the remit of this blog! Now, the good stuff: continuous deployment!
Continuous deployment on AWS #2: Continuous deployment
This is assuming you already have setup Travis CI according to teammate Big George’s Big Blog, here (https://medium.com/full-taxx/how-to-setup-travis-ci-for-a-rails-application-78a453963300).
Thankfully, once you have deployment running anyway, this bit is pretty easy! Most of the following is taken from here: https://docs.travis-ci.com/user/deployment/elasticbeanstalk/
1. Add deployment statement to travis.yml
Find the travis.yml
file in the root folder of your app, then just add the following lines into it:
deploy:
provider: elasticbeanstalk
access_key_id: your_access_key_id
secret_access_key:
secure: your_secure_key
region: your_region
app: your_app_name
env: your_env_name
bucket_name: your_bucket_name
Where:
- your_access_key_id and your_secure_key = another key id / key pair you can make from the AWS console, as we did previously: (log into your AWS account, click on your name (in the top right corner), then select
my security credentials
from the drop-down menu. Click oncontinue to my security credentials
in the pop-up, then select theaccess keys
menu item. Now, selectcreate new access key
, and copy/paste the access key ID and the access key itself into the console.) - your_region = the region you set your app up in. You can find this in the URL for your app (ours, in London, UK, was eu-west-2).
- your_app_name = the name of your app
- your_env_name = the name of the enviroment you are running your app in
- your_bucket_name = the name of the S3 bucket (online storage) that AWS setup to store your source code. To find this, click on the
services
menu at the top of the screen on your AWS console, and selectS3
from the drop-down list. The name of your bucket should be visible.
2. Watch it go!
If everything is setup properly, the next time your push any changes to master, once travis is done checking them, it should deploy automatically to your AWS app!