Continuous Integration with Meteor, Chimp, Galaxy and CircleCI

5 min readDec 3, 2016

Manual testing and deployment is a productivity killer.

Ideally, you want to setup continuous integration where a commit to your repository triggers your tests to run which in turn triggers deployment to staging/production assuming all tests passed. In this way, you focus on writing code and tests instead of spending time running tests and performing manual deployments.

At Unibuddy, this is how we setup our CI on Circle which works with our current Meteor- based application and Chimp tests.

Pre-requisites

This tutorial assumes you already have a Meteor application setup with Chimp which run your Mocha/Cucumber/Jasmine tests. It also assumes you’re familiar with Node and NPM. We deploy with Galaxy, but you can substitute the deployment commands here with anything else you’re using.

Step 1: Create NPM Scripts

Create scripts in your package.json to run your application on npm run dev and also another script so that npm run test will start Chimp.

"scripts": {
     "dev": "meteor run --settings settings-development.json",
     "test": "chimp --ddp=http://localhost:3000 --browser=chrome"
}

Step 2: Create circle.yml config file

Create a circle.yml file in your project’s root directory (same level as package.json). This file is a configuration file which Circle (your CI server) automatically understands. It’s where you script what you want to happen such as setting up the environment for your app, running the app, running tests and deploying.

You can find more about the circle.yml and the various configuration options here.

Step 3: Setup caching and environment

In the beginning you’ll want to ignore caching and just get your CI flow working. However, for a faster pipeline you should think about caching your node_modules directory, .npm and .meteor. This is what is happending under the ‘cache_directories’ section in the code below.

Next, you’ll want to setup your environment which is done in the ‘override’ section. For a Meteor application, this involves installing Meteor (we check if Meteor is installed first, if not then we install — otherwise do nothing). We also install Chimp and Selenium globally. Finally we do an npm install which will install all our dependencies listed in package.json.

NB: If you update package.json you’ll want to clear the cache on Circle before rebuilding. In addition, if you use shrinkwrap, make sure to run npm shrinkwrap if you change your node_modules folder since npm install will install what’s in your shrinkwrap file and not what’s in package.json on Circle.

dependencies:
  cache_directories:
    - "node_modules"
    - "~/.npm"
    - "~/.meteor"
  override:
    # Cache Meteor
    - if [ -d ~/.meteor ]; then sudo ln -s ~/.meteor/meteor /usr/local/bin/meteor; fi
    - if [ ! -e $HOME/.meteor/meteor ]; then curl https://install.meteor.com | sh; fi
    - npm install -g chimp
    - npm install -g selenium-standalone
    - npm install

Step 4: Start application and run tests

Here’s the tricky bit that is very CI specific. You need to run your Meteor application so that it exposes your app to localhost:3000 which Chimp is waiting to connect to and run tests. Critically therefore, we need multithreading — which you don’t normally think about when running your app locally since you would just open multiple terminal windows.

This is why we set the ‘background’ property to true when running our application. Then we wait (sleep) a couple of minutes for the app to build and to be available at localhost:3000. After two minutes, on a different thread, npm run test will execute which will trigger Chimp to connect to app and run all our Cucumber (in our case) tests.

test:
  override:
    - npm run dev:
        background: true
    - sleep 120
    - npm run test

Step 4: Deployment to staging and production

Based on our circle.yml thus far, when you commit — Circle will set up your environment (npm install etc.), run your app and run the tests. If the build succeeds on circle it means the branch you’re currently on and the commit you just made is passing all your tests and is working well. Let’s take it a step further and setup branch based deployment.

Generally, you should have your dev branch in Git synced with your staging server and your master branch synced to production. For commits made to any other branch, you probably don’t want to deploy.

deployment:
  staging:
    branch: "dev"
    commands:
      - echo $DEPLOYMENT_TOKEN > deployment_token.json
      - METEOR_SESSION_FILE=deployment_token.json DEPLOY_HOSTNAME=eu-west-1.galaxy-deploy.meteor.com meteor deploy www.YOUR_STAGING_APP.com --settings settings-development.jsonproduction:
    branch: "master"
    commands:
      - echo $PROD_SETTINGS > settings-production.json
      - echo $DEPLOYMENT_TOKEN > deployment_token.json
      - METEOR_SESSION_FILE=deployment_token.json DEPLOY_HOSTNAME=eu-west-1.galaxy-deploy.meteor.com meteor deploy www.YOUR_PROUDCTION_APP.com --settings settings-production.json

Above, we have the deployment section of our configuration which contains two sections — staging and deployment. We specify the branch dev for staging and master for production. Next, under the ‘commands’ section — we issue deployment commands.

Step 5: Adding Circle environmental variables

If you’re using Galaxy, you will need to setup two environmental variables in Circle.

Firstly, you need to have access to an authorised Meteor account (as setup in Galaxy). In order to create a deployment token, in your terminal on your local dev machine:

METEOR_SESSION_FILE=deployment_token.json meteor login

Follow the prompts (if any) and the result will be the output of file deployment_token.json is your present working directory. Now, go back to Circle — then Project Settings and finally click “Environmental Variables” (link is under Build Settings) on the left sidebar. Click “Add Variable” button, set the name as DEPLOYMENT_TOKEN and copy and paste the contents of deployment_token.json that you just generated. The ‘echo’ command in our circle.yml will extract this environmental variable and recreate deployment_token.json on the Circle server which the final deployment command reads (as METEOR_SESSTION_FILE).

Lastly, you should not have your settings-production.json in your repository (for security purposes). That is why you might notice for production deployment, there’s some extra commands. Find your settings-production.json and copy and paste the entire JSON object as an environmental variable in Circle (as a value) and give it the name PROD_SETTINGS. Similarly, as for the deployment token, the ‘echo’ command extracts the env var and re-creates the settings-production.json which the final production deployment command reads.

The final circle.yml configuration file:

dependencies:
  cache_directories:
    - "node_modules"
    - "~/.npm"
    - "~/.meteor"
  override:
    # Cache Meteor
    - if [ -d ~/.meteor ]; then sudo ln -s ~/.meteor/meteor /usr/local/bin/meteor; fi
    - if [ ! -e $HOME/.meteor/meteor ]; then curl https://install.meteor.com | sh; fi
    - npm install -g chimp
    - npm install -g selenium-standalone
    - npm install
test:
  override:
    - npm run dev:
        background: true
    - sleep 360
    - npm run testdeployment:
  staging:
    branch: "dev"
    commands:
      - echo $DEPLOYMENT_TOKEN > deployment_token.json
      - METEOR_SESSION_FILE=deployment_token.json DEPLOY_HOSTNAME=eu-west-1.galaxy-deploy.meteor.com meteor deploy www.YOUR_STAGING_APP.com --settings settings-development.jsonproduction:
    branch: "master"
    commands:
      - echo $PROD_SETTINGS > settings-production.json
      - echo $DEPLOYMENT_TOKEN > deployment_token.json
      - METEOR_SESSION_FILE=deployment_token.json DEPLOY_HOSTNAME=eu-west-1.galaxy-deploy.meteor.com meteor deploy www.YOUR_PRODUCTION_APP.com --settings settings-production.json

It can be frustrating to setup but once you’ve got it working, you’ll never go back to manual deployments!

Code. Commit. Sit back and watch your CI server do the rest.