Parallelizing Builds In Travis CI

Parth Shandilya
HackerNoon.com
Published in
3 min readMar 18, 2018

--

Ember JS + Python

A little background: over the last few months, I have been contributing in open source organization FOSSASIA, where I’m working on a project called BadgeYaY. It is a badge generator with a simple web UI to add data and generate printable badges in PDF.

Badgeyay project is now divided into two parts i.e front-end of Ember JS and back-end with REST-API programmed in Python. Now, one of the challenging job is that, it should support the uncoupled architecture. It should therefore run tests for the front-end and backend i.e, of two different languages on isolated instances by making use of the isolated parallel builds.

In this blog, I’ll be discussing how I have configured Travis CI to run the tests parallely in isolated parallel builds in Badgeyay in my Pull Request.

First let’s understand what is Parallel Travis CI build and why we need it. Then we will move onto configuring the travis.yml file to run tests parallely. Let’s get started and understand it step by step.

Why Parallel Travis CI Build?

The integration test suites tend to test more complex situations through the whole stack which incorporates front-end and back-end, they likewise have a tendency to be the slowest part, requiring various minutes to run, here and there even up to 30 minutes. To accelerate a test suite like that, we can split it up into a few sections utilizing Travis build matrix feature. Travis will decide the build matrix based on environment variables and schedule two builds to run.

Now our objective is clear that we have to configure travis.yml to build parallel-y. Our project requires two buildpacks, Python and node_js, running the build jobs for both them would speed up things by a considerable amount.It seems be possible now to run several languages in one .travis.yml file using the matrix:include feature.

Below is the code snippet of the travis.yml file for the Badgeyay project in order to run build jobs in a parallel fashion.

sudo: required
dist: trusty
# check different combinations of build flags which is able to divide builds into “jobs”.
matrix:
# Helps to run different languages in one .travis.yml file
include:
# First Job in Python language.
— language: python3
apt:
packages:
— python-dev
python:
— 3.5
cache:
directories:
— $HOME/backend/.pip-cache/

before_install:
— sudo apt-get -qq update
— sudo apt-get -y install python3-pip
— sudo apt-get install python-virtualenv
install:
— virtualenv -p python3 ../flask_env
— source ../flask_env/bin/activate
— pip3 install -r backend/requirements/test.txt — cache-dir
before_script:
— export DISPLAY=:99.0
— sh -e /etc/init.d/xvfb start
— sleep 3
script:
— python backend/app/main.py >> log.txt 2>&1 &
— python backend/app/main.py > /dev/null &
— py.test — cov ../ ./backend/app/tests/test_api.py
after_success:
— bash <(curl -s https://codecov.io/bash)
# Second Job in node js language.
— language: node_js
node_js:
— “6”
addons:
chrome: stable
cache:
directories:
— $HOME/frontend/.npm
env:
global:
# See https://git.io/vdao3 for details.
— JOBS=1
before_install:
— cd frontend
— npm install
— npm install -g ember-cli
— npm i eslint-plugin-ember@latest — save-dev
— npm config set spin false
script:
— npm run lint:js
— npm test

Now, as we have added travis.yml and pushed it to the project repository. Here is the screen-shot of passing Travis CI after parallel build jobs.

The related Pull Request of this work is https://github.com/fossasia/badgeyay/pull/512

With that, I have reached the end of our discussion on Parallelizing Builds In Travis CI. I wrote this post as a solution to this issue in BadgeYaY project. If you liked this post, consider having a look at my other work on GitHub 🙂.

Source: Travis CI documentation

PS: Constructive criticism is very much wanted. 🙂

--

--

HackerNoon.com
HackerNoon.com

Published in HackerNoon.com

Elijah McClain, George Floyd, Eric Garner, Breonna Taylor, Ahmaud Arbery, Michael Brown, Oscar Grant, Atatiana Jefferson, Tamir Rice, Bettie Jones, Botham Jean

Parth Shandilya
Parth Shandilya

Written by Parth Shandilya

Engineer | Tech Writer @HackerNoon, @freeCodeCamp

No responses yet