How we use CircleCI with Git and DigitalOcean to streamline our deployments

❤

I’ll preface by saying we love CircleCI. If you’re into streamlining development and deployment workflows, look no further. We’re still using their generous free plan, but if a day comes when we’re lucky enough to be managing a number projects simultaneously which requires more than the free allowance, you can be sure we’ll be forking out.

Step 1 — Sign up for CircleCI and link your GitHub/BitBucket account(s)

Pretty simple. Head to https://circleci.com and sign up for their free plan.

Step 2 — Choose the project that you would like CircleCI to continuously deploy for you

Once you’ve added your preferred Git accounts to CircleCI, you’ll be able to see all your projects under the “Add Projects” section on the left hand side. Note that you might have to switch between Git accounts in the top left if all of your repos aren’t in the same place.

If you have a ton of Git accounts like me you’ll have to select the one that your project belongs to

Once you’ve found your project in the list, hit “Set Up Project.”

Step3 — Add a CircleCI config file to your repository

You’ll receive a sample file that you can plonk in your repo at this point, under .circleci/config.yml. You can set up this file however you like, for now we’ll use the default one given by CircleCI but we’ll be coming back to it. Commit and push the changes, then tell CircleCI to start watching the project. If you’ve done everything correctly, your config file should be interpreted by CircleCI and your project should start building.

Step 4 — Configure your VPS

To perform automatic deployments, CircleCI is going to need to log in to our server and pull the latest code from our git repo. The first thing we’ll do is generate an SSH key for CircleCI to use to log in. Run the following command on your own machine:

$ ssh-keygen -m PEM -t rsa -f ~/.ssh/id_rsa_circleci

Now we’re going to create a new user for CircleCI to log in as, so that we can appropriately permission it to deploy our project. First, log into your VPS as a user with root permissions and run the following commands to generate a user (you made need to use sudo):

$ useradd -m -d /home/circleci -s /bin/bash circleci
$ mkdir /home/circleci/.ssh
$ touch /home/circleci/.ssh/authorized_keys

Now we’re going to add the key we created on our own machine to the list of authorised SSH keys for the CircleCI user on the droplet. This will allow us to log in as the circleci user without needing to provide a password. On our own machine, run the following to add the public key to our clipboard:

$ pbcopy < ~/.ssh/id_rsa_circleci.pub

Then, on the server, paste the key into the authorized_keys file that we made earlier:

$ pbpaste > /home/circleci/.ssh/authorized_keys

Now we should be able to ssh into our server as the circleci user. Test this out from your machine by running the following:

$ ssh circleci@my.droplet.ip -i ~/.ssh/id_rsa_circleci

You should be connected and find yourself in the /home/circleci directory.

Step 5 — Configure CircleCI to use the SSH key we just made

Now that we’ve verified our SSH key works, we can tell CircleCI about it. Navigate to your project’s settings and find “SSH Permissions” on the left hand side. You should be presented with a view which allows you to add an SSH key, so click that button.

CircleCI is asking for our private key here. The private key is used to prove that we own the public key which is stored in the authorized_keys file on the server. Since we generated the keypair on our own machine, go back to the terminal and run the following to copy the private key to your clipboard:

$ pbcopy < ~/.ssh/id_rsa_circleci

Now paste this into the dialog box presented by CircleCI, leave the hostname blank and save the key.

Step 6 — Create a deploy script on our server

This step will vary depending on your project’s setup, but we’ll go through the process as if we were pulling updates to our static website which is served by apache, hopefully you’ll be able to see the rough idea and configure your own deploy script for your own project(s).

Log into our server as the circleci user again:

$ ssh circleci@my.droplet.ip -i ~/.ssh/id_rsa_circleci

Then create a deploy script in your home directory:

$ touch ./deploy_project.sh
$ chmod u+x ./deploy_project.sh

Inside the script, we’re going to add the following, but your project may differ (you may need to checkout a specific branch/build the site/install deps/etc.):

cd /var/www/html
git pull
npm i

N.B. You may need to change the permissions of the /var/www/html directory so that your circleci user can edit the contents. If you do, something like this should do the trick:

$ sudo chown -R circleci:circleci /var/www/html

Step 7 — Configure your server to use a Git SSH key

In order to avoid the password prompt when pulling a Git repository, we’re going to add the repository via SSH.

We need to log into our server again as the circleci user.

$ ssh circleci@my.droplet.ip -i ~/.ssh/id_rsa_circleci

And then create an SSH key which we can add to either our GitHub/BitBucket account (depending on where the repository is hosted). You may also choose to use an SSH key which you already have, though generally it’s good practise to create a new SSH key per machine.

$ ssh-keygen -t rsa -f /home/circleci/.ssh/id_rsa_git

Now we’ve got another keypair, we can tell our Git service about the public key. We don’t need to give our private key away here, since we’re trying to authenticate our own machine with the Git remote. Print the public key on the server’s terminal and then copy it to your clipboard:

$ cat /home/circleci/.ssh/id_rsa_git.pub

Head over to your provider of choice and add the public key. Here are instructions for GitHub and BitBucket.

We’re also going to clone our repository via SSH into the directory that we want to serve it from. Again, here we’re assuming that the project is a static website served by apache/nginx, but your project may be completely different!

Make sure you select SSH from the dropdown, we don’t want to use the HTTPS URL!

cd /var/www/html
git clone <Project’s SSH URL> . # Make sure you include the period

Step 8 — Tell CircleCI how to deploy the project

Remember the config.yml file that CircleCI gave us a while ago and I said we would come back to it? It’s time to go back to it.

We’re going to add another job under the jobs level with the following content.

deploy-prod:
  docker:
  # specify the version you desire here (you might not want node)
  - image: circleci/node:7.10
  steps:
    - checkout
    - run: ssh -oStrictHostKeyChecking=no -v $DROPLET_USER@$DROPLET_IP “./deploy_project.sh”

Note the $DROPLET_USER and $DROPLET_IP values here, they’re environment variables that we need to tell CircleCI about.

Head over to the project’s settings and navigate to “Environment Variables”. Add two, one for DROPLET_USER (value: circleci , the user that CircleCI should log into our server as) and DROPLET_IP , the IP address of your droplet.

Then we’re going to add a workflow at the root level of the file:

workflows:
  version: 2
  build-and-deploy:
    jobs:
      - build
      - deploy-prod:
          filters:
            branches:
              only:
                - master
          requires:
            - build

You can infer what this is doing by reading the code here, but essentially we’re telling CircleCI to only run the deploy-prod job if the branch we’ve updated is the master branch. This is a neat way to ensure we’re not pushing development builds to our server, but you could remove this filter if that’s what you wanted. You can make multiple workflows if you have a bunch of environments. I find that this approach works really well with git flow.

Save the changes, then commit and push.

Step 9 — Check if everything works!

Push some code to your master branch and check that the changes are reflected in your production build after CircleCI is finished! If you encounter any errors, the CircleCI console should do a decent job of pointing them out.

Step 10 — Pop some corks because you’ve just saved yourself hours in the future