Gitlab CI/CD Crash Course
Beginner’s guide for integrating test, build and deploy to your Gitlab workflow
What is CI/CD?
Continuous Integration (CI) is a term usually used to describe an automated process within software development, where developers can make sure that what they’re adding to the software does not break whatever is currently working.
While CD, also known as Continuous Delivery or Continuous Deployment, takes CI a step further. It aims for automated delivery of new features for the customer, acceleration feedback loop.
I see CI/CD as a part of the DevOps culture as described by Martin Fowler. It automates testing and deployments, freeing people’s time to focus on other things. It also promotes stronger collaboration as developers have to stick with small code changes and sync up frequently with the rest of the team.
Gitlab as a complete DevOps lifecycle application provides its users with an integrated CI/CD feature which is managed through a single file placed at the repository’s root.
In a nutshell, the users will put the CI/CD configuration in a file named
.gitlab-ci.yml and every time a code change is pushed to the Gitlab repository, Gitlab with run the configured CI/CD script.
Scripts configured for Gitlab CI/CD will be executed by the Gitlab runner. Gitlab runner is an open-source project with various features on how to run the CI/CD script. Gitlab runner has several executor options such as SSH, Docker, and VirtualBox, to fit the users' needs.
Pipelines are the top-level component of continuous integration, delivery, and deployment. A pipeline will be created when a new push is made to the Gitlab repository. The pipeline will immediately run when there is an idle Gitlab runner or be put in a queue if all available runners are being used by someone else.
A pipeline will consist of jobs and stages.
Jobs is something that defines what will be run. Below is an example of
.gitlab-ci-yml file with one job named test. We also specify that the job will run with
ruby-2.6.3 docker image. By default, when running a job Gitlab will spawn a docker container (image can be specified as shown below), clone your repository inside the container, and run the specified script.
- gem install bundler
- bundle install
- bundle exec rails spec
The pipeline of with CI configuration shown above will look like this:
Stages serve as categorization of jobs. It defines when and how a job will be run. By default, a pipeline will have three stages: build, test, and deploy. Having default stages does not mean the users have to create a job for each stages, for stages with no jobs will simply be ignored. Below we show a configuration with build and test stages:
- gem install bundler
- bundle installtest:
- bundle exec rails specbuild:
- bundle package
If we don’t specify a stage in a job, Gitlab will automatically put it under
test stage. Here we specify two jobs,
test which will run on the test stage and
build which will run on the build stage. We also use
before_script which specify commands that we want to run before each job. The pipeline of the configuration above will look like this:
As mentioned previously, Gitlab will have
deploy stage and the unused stage (in the current case is deploy) will be ignored.
Gitlab CI/CD also support the creation of custom stages
Above we tried to switch the default order of
build stage and we added
pre-build stage. The pipeline will look like this:
Does it always run after each other? By default, the next stage will run after the previous one has succeeded. So when the previous failed, the next stage will not run. What if we want to allow some jobs to fail? Gitlab will allow the next stage to run if the failing job is specified with
What if I want a job not being run automatically? For that, Gitlab provides the option to specify when to run the job. Quoting from the docs, the options provided are
If you’re new at this, you might get frustrated when editing your
.gitlab-ci.yml, pushing it to Gitlab and see that there is actually a syntax error in your configuration. Forcing you to add dirty commits just to fix the syntax error. To evade this I can suggest some methods:
- Validate the syntax of your configuration before pushing, Gitlab is nice enough to provide you the way, use it.
- Configure your own runner, and test with it first before pushing to Gitlab.
Downloading packages may be a process that takes a long time within your pipeline. To make it faster you can cache the files so that you don’t have to redownload everything everytime you pipeline runs. Additional note on caching, don’t forget to clear the cache when you update some things you’re caching and don’t use cache to pass files between separate jobs (read the next section for more about this). Another way to address the problem is to create your own docker image tailored for your own needs.
You might want to use Gitlab CI/CD to automate packaging and deployment for your application. In doing this, you might encounter the need to pass files generated in one job to another job. To do this learn you should learn how to use artifacts.
When using at a larger scale (maybe deploying Gitlab for your institution/company) you might expect a queue if you don’t configure your runner wisely, look more into shared, specific, and group runner to address that.