Apply Continuous Integration for Ruby project by using CircleCI
Most software developers might have experienced this kind of situation: The more members involved in the software project, the more chance that errors appear when new code integrates into the project. To avoid this kind of problem, The concept of Continuous Integration(CI) comes out and it becomes popular to be applied to software project nowaday. What is Continuous Integration(CI)? According to the definition of Continuous Integration(CI) from Atlassian website, we could know it is a good practice for software projects to keep the new code’s correctness before integration into the project.
Continuous integration (CI) is the practice of automating the integration of code changes from multiple contributors into a single software project. It’s a primary DevOps best practice, allowing developers to frequently merge code changes into a central repository where builds and tests then run. Automated tools are used to assert the new code’s correctness before integration.
There are many tools to fulfill Continuous integration (CI) in software projects, for example, Jenkins, TeamCity, Bamboo, Buddy, and CircleCI... Every tool has its pros and cons. This article chooses CircleCI to be an example to show how to apply Continuous Integration to a Ruby project. Below is the outline of this article. There are 5 sessions to introduce the usage of CircleCI.
Outline
．Getting started with CircleCI on your Github repo
．Implement CircleCI in Ruby project
．Setup Database in CircleCI
．Apply reusable command with parameters in CircleCI config
．Debugging CircleCI error
．ReferenceGetting started with CircleCI on your Github repo
To start using CircleCI, First of all, you should sign up with GitHub or BitBucket. In this article, we are using Github to signup. After signing up and logging in, you could see on the project page all your GitHub projects will list.
Before clicking the Set Up Project Button from the project you want to apply CircleCI, you could create a .circleci the folder at the root of your project, and create a config.yml file inside the .circleci folder. With this .circle/config.yml , after setting up the project with CircleCI, every time when there is a new change pushed to the Github repo, CircleCI will read the config.yml and execute the jobs defined in the config.yml
The content of config.yml you may use the example config share in CircleCI Configuration Introduction testing first. The example config is as below. We could see there is one job buildin the config, and which docker image used in the job environment was also defined, so when the project is set up with CicleCI, the environment for the build job will be prepared every time the job runs. In the config, the command which is executed when the job runs is also defined. It will execute two commands: print Hello Wolrd! and then print This is the delivery pipeline
# CircleCI config.yml exampleversion: 2.1
jobs:
build:
docker:
- image: alpine:3.15
steps:
- run:
name: The First Step
command: |
echo 'Hello World!'
echo 'This is the delivery pipeline'
After committing the config.ymland push it to Github, you could click on the Set Up Project Button, and there will be a pop-up window to let you choose the way to use your config file. We could just choose the fastest one and click set up the project.
After setting up the project, there is a pipeline shown in your dashboard. All the jobs defined in the config.yml will show. If there is a green tick shown beside the jobs, which means all the commands in the job are executed successfully.
If there is a red exclamation mark, it means the job fails.
We could click on the jobs to see more information. In our example, if we click on build, there will show all the detailed results of this job. We could click on steps and there will be a step call The First Step which we define in the config.yml
Click on the The First Step you could see the two commands in this step execute.Hello World! and This is the delivery pipeline print on the console, so if we want to implement another command, just update it in the steps of config.yml , commit it and push it to the Github repo.
If we need multi jobs run and some jobs should run after some jobs are done, the workflowscould be implemented in config.yml . Below is the example config in CircleCI document. In the example, there are three jobs defined: Hello-World, Fetch-Code, and Using-Node. In the workflows part, therequires could let CircleCI know the job needs to execute only when which job is done. We could see Fetch-Code could execute only the first job Hello-World is done and Using-Node could execute only Fetch-Code is done.
# CircleCI config.yml exampleversion: 2.1
jobs:
# running commands on a basic image
Hello-World:
docker:
- image: cimg/base:2021.04
steps:
- run:
name: Saying Hello
command: |
echo 'Hello World!'
echo 'This is the delivery pipeline'
# fetching code from the repo
Fetch-Code:
docker:
- image: cimg/base:2021.04
steps:
- checkout
- run:
name: Getting the Code
command: |
ls -al
echo '^^^Your repo files^^^'
# running a node container
Using-Node:
docker:
- image: cimg/node:17.2
steps:
- run:
name: Running the Node Container
command: |
node -v
workflows:
Example-Workflow:
jobs:
- Hello-World
- Fetch-Code:
requires:
- Hello-World
- Using-Node:
requires:
- Fetch-Code
After committing the config.yml and push it to Github, we could see in the CircleCI dashboard there is a pipeline, and there are three jobs which are the jobs we define in config.yml
Click on the success button, there is the detail of the workflow. It shows the jobs in the order just as we define in config.yml
In the job Fetch-Code, there are two steps defined. The first one is checkout . It fetches the code from your Github repo when it executes, so that’s why step two could command ls -al to list all the contents of the Github repo.
# fetching code from the repo
Fetch-Code:
docker:
- image: cimg/base:2021.04
steps:
- checkout
- run:
name: Getting the Code
command: |
ls -al
echo '^^^Your repo files^^^'Click on the Fetch-Code to see detail, we could see the two steps: Checkout code and Getting the Code.
In the Checkout code part, the code has been fetching from your git repo and it is on the branch of origin/master
In the Getting the code part, CircleCI execute ls -al and list all the files and folders in the first layer of your repo.
Every time we do a code change in our project and push it to the Github repo, CircleCI will run the jobs and execute all the commands in the steps which were defined in config.ymlautomatically. There is clear instruction on how to write config.yml to implement Continuous Integration in your software project in CircleCI document. People who are interested in learning more could see the CircleCI config introduction and take the example to get familiar with how CircleCI works with the Github repo.
Continue｜Back to OutlineImplement CircleCI in Ruby project
To implement CircleCI in Ruby, our aim is to run all the RSpec tests automatically every time when there is a code change and pushed to the Github repo, so CircleCI needs to execute run RSpec. The example Rails project used in this article could be found here. The ruby version is 2.7.1 and the database is using SQLite. Before running RSpec in CircleCI, the test environment should be prepared, so there will be two jobs defined in the .circle/config.yml . The first is for building the environment, the second one is for running the testing. We will use workflows to define the order and requirements of the jobs. The example config is as below.
version: 2.1orbs:
ruby: circleci/ruby@1.1.0jobs:
build:
docker:
- image: cimg/ruby:2.7-node
steps:
- checkout
- ruby/install-deps
test:
docker:
- image: cimg/ruby:2.7-node
steps:
- checkout
- ruby/install-deps
# Run rspec
- ruby/rspec-testworkflows:
version: 2
build_and_test:
jobs:
- build
- test:
requires:
- build
Because we will install ruby in two jobs, we implement orbs to reuse ruby install steps which already defined in CircleCI, so that we could use ruby/install-deps to call all the commands in every job to install ruby in CircleCI.
More information about orbs could be found in CircleCI document Orbs Introduction. Below is the definition of orbs in CircleCI page.
Orbs are reusable snippets of code that help automate repeated processes, accelerate project setup, and make it easy to integrate with third-party tools.
Except prepare the config.yml , the Gemfile also need to update, because CircleCI needs gem 'rspec_junit_formatter' to execute rspec. gem 'rspec_junit_formatter' need to be added in group :development, :test of Gemfile like below.
group :development, :test do
....
gem 'rspec_junit_formatter'
....
endAfter the config.yml and Gemfile , we could see the pipeline with two jobs in CircleCI dashboard after committing the change and pushing it to the Github repo.
Clicking on the build job, we could see after Checkout code there is three-step: Restoring cache, Bundle install, Saving cache, and clicking on test job, also could find these three jobs after Checkout code. These three steps are the steps that will run when executing ruby/install-deps
In the testjob detail page, we could see the RSpec is executed after Saving cache.
Clicking on the step and we could see all the RSpec tests have been executed. Because there are only two test cases in this project, only two test results show, but the result would be just like execute bundle exec rspec in your local environment. And every time there is now code change and push to the Github repo, bundle exec rspec will be triggered to check if any test case fails.
More config settings for Ruby project could be found in CircleCI document Language Guide: Ruby. There is more example to show how to implement CircleCI in Ruby project.
Continue｜Back to OutlineSetup Database in CircleCI
If the database in your project is not SQLite, it also could set up the database environment in CircleCI. Just need to define the docker image of the database and environment parameters in the config.yml . Below is the example config of using the PostgreSQL database. In the steps, after running ruby/install-deps , will continue to run the database setup command.
version: 2.1orbs:
ruby: circleci/ruby@1.1.0
jobs:
build:
docker:
- image: cimg/ruby:2.7-node
steps:
- checkout
- ruby/install-deps
test:
# parallelism: 3
docker:
- image: cimg/ruby:2.7-node
- image: circleci/postgres:9.5-alpine
environment:
POSTGRES_USER: circleci-demo-ruby
POSTGRES_DB: rails_blog_test
POSTGRES_PASSWORD: ""
environment:
BUNDLE_JOBS: "3"
BUNDLE_RETRY: "3"
PGHOST: 127.0.0.1
PGUSER: circleci-demo-ruby
PGPASSWORD: ""
RAILS_ENV: test
steps:
- checkout
- ruby/install-deps
- run:
name: Wait for DB
command: dockerize -wait tcp://localhost:5432 -timeout 1m
- run:
name: Database setup
command: bundle exec rails db:schema:load --trace
# Run rspec in parallel
- ruby/rspec-testworkflows:
version: 2
build_and_test:
jobs:
- build
- test:
requires:
- build
In the test job running result of the example config, we could see the docker image of PostgreSQL database is using and the two steps to set up the database, Wait for DB and Database setup, have been executed.
Clicking on these two steps, we could see the log of table creation in the PostgreSQL database.
More database config examples could be found in CircleCI document Database Configuration Examples.
Continue｜Back to OutlineApply reusable command with parameters in CircleCI config
If there is a need to run a command in different jobs or steps but have to pass different parameters, CircleCI supports us to define custom commands with parameters. We could print greeting words before running all the jobs, and part of the greeting words depend on what we pass to the parameter in the different jobs. The example config is as below. There is a greeting command defined with a parameter to which has a default value world , and in the build job, the greeting command is in the steps with the to value : build . The greeting command is also shown in the test job and the to value is test .
version: 2.1orbs:
ruby: circleci/ruby@1.1.0commands: # a reusable command with parameters
greeting:
parameters:
to:
default: "world"
type: string
steps:
- run: echo "Hello <<parameters.to>>"
jobs:
build:
docker:
- image: cimg/ruby:2.7-node
steps:
- greeting:
to: "build"
- checkout
- ruby/install-deps
test:
docker:
- image: cimg/ruby:2.7-node
steps:
- greeting:
to: "test"
- checkout
- ruby/install-deps
# Run rspec in parallel
- ruby/rspec-testworkflows:
version: 2
build_and_test:
jobs:
- build
- test:
requires:
- build
In the result of the example config, both in build and test jobs, the greeting command are all executed and they print out the greeting words with what we pass to the parameter to
There are more details and examples about the reusable config in CircleCI document. That information could be found in CircleCI Reusable Config Reference Guide.
Continue｜Back to OutlineDebugging CircleCI error
If the jobs fail, we could click the jobs and see more information. Most of the time, the info would be clear enough for us to debug. Like the bundle install error below, it clearly shows the failure is caused by the wrong version of the gem.
But sometimes the error message might not be clear enough for us to know the root cause of failure, another way to debug is by using Rerun Job with SSH.
When clicking on the Rerun Job with SSH , CircleCI will prepare a remote virtual machine that is just the same as the environment of your failed job executed. We could see in the step Wait for SSH session that there is a command provided for us to access the machine.
ssh -p 64535 3.95.238.182
By typing the command in your local console, you could successfully login to the machine.
Go to the project folder, and we could see the content is just the same as your Github project, then we could run the command in our failure job in CircleCI to see more information.
Back to OutlineReference
- Configuration Introduction
- Language Guide: Ruby
- Reusable Config Reference Guide
- Orbs Introduction
- Database Configuration Examples
What is Continuous Integration and Why it is Important?
How to set up a continuous integration pipeline for a Rails dual boot
Continuous Integration and Deployment for Rails using CircleCI
Top 7 Continuous Integration Tools for DevOps
