Example of CI/CD stack

When I started to work at TRAXxs (we are doing some pretty cool stuff there btw :) ), there was a lot of things to put in place to setup our back-end / front-end layers. The cool thing was the possibility to select the technologies we will use… Let’s go Docker, Node.js, Mongo + other cool stuff then (a little bit of Clojure + some Go, as they are languages that Im also very interested in).

At TRAXxs, we are working on connected insole devices. To keep it simple, on the server side (the part I’m in charge) we collect the data, make some analysis and expose it to the clients via api and web front-end. There is, obviously, much more than this but this is a cool summary.

Service architecture

We have several services (do not know if I should call them microservices…) and I’d like them to follow the same development flow. The idea is to setup a simple Continuous Integration / Continuous Deployment pipeline. Not a complete one for now on as I’m not an expert in this domain, but at least something that gives us the confident so we can have code pushed, tested and deployed automatically several times a day. Of course, this pipeline will be improved in the future.

The purpose of this article is to give a quick overview of what we have done so far. We will not go into great details here.

There are stuff I really want to be part of the picture, and this will contribute to the list of the requirements :)

• the pipeline must be triggered by PR merge
• tests must be automated and run on a SaaS service
• usage of Docker Hub as our image registry
• usage of Docker Cloud to manager our deployments
• usage of DigitalOcean as cloud provider

Tools selection

A lot of tools are available out there.

a) For the versioning part, I’m using git / GitHub on a daily basis, so I did not hesitate much when it came to select a tool in this area. I’ll go for GitHub then.

b) For the testing part, the choice is thougher. I’ve tested most of the different tools listed here and for ease of use and price consideration (it offers 1500 free minutes of build per months) I’ve selected CircleCI. The setup is quite simple, basically a circle.yml file needs to be added to the codebase of the service, this file describing the different steps that needs to be performed.

For a simple project, circle.yml is not even needed as CircleCI can infer the language. It can for instance detect a Node.js project if a package.json file is present (well, I think this is how it works…). For project a little bit more complexe, a circle.yml needs to be provided. Below is an example of a circle.yml file used to test and deploy one of our web front-end.

machine:
  node:
    version: 4.4.0
  services:
    - redis
dependencies:
  override:
# Cache npm deps
    - npm install
# Cache bower deps
    - npm install -g bower
    - bower install
# Build assets
    - npm install -g grunt-cli
    - grunt test
# Cache Selenium & ChromeDriver
    - ./node_modules/.bin/chimp --path=content # there are no tests in content
test:
  pre:
    - mkdir -p $CIRCLE_TEST_REPORTS/cucumber
override:
    - npm test:
        environment:
          CUCUMBER_JSON_OUTPUT: "$CIRCLE_TEST_REPORTS/cucumber/chrome.cucumber"
deployment:
  production:
    branch: master
    commands:
      - "curl -H \"Content-Type: application/json\" --data '{\"source_type\": \"Branch\", \"source_name\": \"master\"}' -X POST https://registry.hub.docker.com/u/traxxs/web/trigger/a34c2122-cb12-126e/"
  staging:
    branch: develop
    commands:
      - "curl -H \"Content-Type: application/json\" --data '{\"source_type\": \"Branch\", \"source_name\": \"develop\"}' -X POST https://registry.hub.docker.com/u/traxxs/web/trigger/453ea32a-ae2c-1ac3/"

The file above defines several configurations / actions:

• Node.js version to use (4.4.0)
• service needed (redis)
• compilation of the dependencies (Node.js + Bower)
• testing phase (using chimp / cucumber / selenium)
• deploy phase (target image creation on Docker Hub)

c) For the Ship / Run step, it's a no brainer, Docker is the platform of choice.

d) For the cloud provider, I've selected DigitalOcean as I really like working on their Droplet. In a near future, I'll need to study Amazon AWS.

Required steps

We'll not go into the details of all the setup, basically only a couple of steps are required.

• Create a repository on GitHub
• add Dockerfile in the application codebase
• create Docker account (used for both Hub and Cloud)
• link GitHub and CircleCI
• A little bit of configuration

CI/CD flow

The first version of the pipeline put in place with the selected components is the following one.

Several things are missing here, for instances the creation of a container will the whole application to perform some real integration tests. This pipeline will be enhanced in a near future.

I would love to have your feedback on this topic.