Migrating from Jenkins to GitLab CI : DSL side by side
At Spatial Vision, we have been using Jenkins for Continuous Integration, building, testing and deploying web apps, native mobile apps, APIs and AWS resources to target environments.
Jenkins manages over 50 projects and nearly 500 jobs including active development builds and on-going maintenance tasks.
Today, we’d like to share our story on migrating from Jenkins to GitLab CI, by comparing the DSL of each platform so you can understand the differences between them and it gives you a quick introduction of GitLab CI for Jenkins users.
- Our Jenkins
- Jenkins cons (why we looked for an alternative)
- GitLab CI
- Convert Jenkins DSL to GitLab CI DSL
- Lessons Learned
Our Jenkins
We have a total of 4 Jenkins instances, running on an internal Ubuntu LXC containers, and automating CI processes using a number of build machines.
The reason we run multiple Jenkins instances is that Jenkins could become slower in listing, viewing and editing of jobs when you add a lot of jobs.
We provide managed services for our clients using AWS based infrastructure and each project typically has a webapp/native mobile apps and APIs (Development/QA/UAT/Production environments).
The backed services are created and updated by AWS CloudFormation and AWS CLI.
Jenkins Jobs
The following are common Jenkins Jobs and they are executed based on the code change or periodic schedule.
- Build HTML 5 web applications using Node.js and Angular
- Build native mobile applications (Ionic, NativeScript, ReactNative) in a Mac mini build machine
- Deploy to Google Play and iTunes using Fastlane/MS App Centre
- Build and test APIs (Node.js, Kotlin/Java) on a Dockernized build machine
- Build and push docker images to Docker Registry (GitLab built-in registry and Docker Hub)
- Deploy AWS resources including S3, CloudFront, Lambda, EC2/ECS, Fargate, RDS etc.
- Deploy Docker containers for Docker compose based APIs
- Create/update database instances including AWS RDS (PostgreSQL and MS SQLServer)
Jenkins agents
These jobs are executed on an internal build machines (Ubuntu, Mac mini and Windows) and AWS EC2 instances in which Jenkins Agents are installed for deployment jobs.
Jenkins cons (why we looked for an alternative)
Jenkins has been around for many years and we see it as one of the best model of open source based platform. It has many plugins that are ready to perform a particular job. However we’ve found some issues over a period of time and have been looking for an alternative tools for a quite while.
- Although Jenkins project is still active, we feel the community is shrinking and some issues never get resolved and there are many plugins that are outdated
- Jenkins Configurations, including Global and project level, are not easily reusable as these are managed within the Jenkins UI
- Jenkins introduced pipeline (Groovy script based ) which could be very complicated for a very simple job and you need a JVM/Groovy for a project even you might only want to build a NPM package
- There is a configuration plugin that lets you create ‘Configuration as Code’ and it offers yaml syntax but it only supports a small set of plugins and you might need to overwrite the global configuration for a project, which might be harder to manage when you have a large number of global configuration established
- We feel Jenkins web application user experience is quite outdated and make you click so many links to view what you’re looking for such as Console output for a particular build
- Jenkins now offer CDF, Jenkins X, Tekton and Spinnaker, that are promoted as new generation Continuous Delivery platforms (we rather want a CI/CD platform that works well)
GitLab CI
We’ve been using Self-Managed GitLab for our code repository. GitLab provides much more than just a code repository, such as a docker registry, error tracking, wiki, issue tracking, CI etc.
GitLab CI has been around for a couple of years and it has become one of the most popular CI tools in the community. GitLab products have a great documentation and their feature development is so fast and we get new features so frequent and they are generally very useful.
We’ve tried on GitLab CI by migrating some existing Jenkins jobs including building and deploying API, webapp and native app.
We’ll walk through the migration process by comparing the DSLs used in both platforms.
Convert Jenkins DSL to GitLab CI DSL
Our user story
“Build and Push Kotlin based API Docker image and deploy to a development environment”
Node/Agent/Slave vs GitLab Runner
First of all, we need a machine to run a job, which can be any platforms including Linux, Mac and Windows.
Jenkins Agent
Jenkins Agent must be installed on a host machine and you specify the work space (remote directory), labels and availability. You can decide the number of executors in order to run jobs concurrently.
GitLab Runner
For GitLab, you need to install a GitLab Runner on a host machine and then register runners. You can register any number of Runners (Executors) on a host machine in order to run jobs in parallel.
GitLab Runner has a set of arch types, including SSH, Shell, Parallels, VitualBox, Docker or even Kubernetes. We like this feature as we often use a Docker container for a build and it is naturally supported:
For building an API image, we use Shell executor to build a Docker image and push it to GitLab registry.
Manage node vs Config.toml
Jenkins only provides the Agent configuration via UI, called ‘Manage Node’. GitLab Runners can be configured via UI or using a config.toml file installed on the host of Runners.
Labels vs Tags
Labels in Jenkins describes the capabilities of the Agent so when you run a job, you can specify ‘what capability is required’ instead of ‘what machine should be used’. GitLab CI calls it Tags, which seems more natural DSL.
Workspace vs builds & cache
We normally create a Jenkins user for a Jenkins Agent and allocate a home directory, which is the same in GitLab Runners.
Jenkins creates a directory called workspace. When you create a new job, a sub directory will be created in workspace in where the code is cloned and built.
GitLab Runner creates ‘builds’ directory and each Runner has its own sub directory inside it. Runners cache any artefacts (specified in gitlab-ci.yml) in ‘cache’ directory.
GitLab has a concept of group/sub groups to bundle related to repositories and Runners replicate the group/sub group structure inside the builds directory. e.g. builds/my-runner/my-project-group/my-api
Create a new job
When a Runner has been installed and registered, we can then create a CI job.
New Item vs The repository
In Jenkins, you need to create a ‘New Item’ to create a job AFTER you’ve created a repository somewhere (GitLab, GitHub, Bitbucket etc).
GitLab CI is immediately available for EVERY repository you create on GitLab.
Configure vs .gitlab-ci.yml
In Jenkins, you specify the build details in ‘Configure’ menu, which has a total of 6 tabs including ‘General’, ‘Source Code Management’, ‘Build Triggers’, ‘Build Environment’, ‘Build’ and ‘Post-build Actions’.
We normally use ‘Copy from’ feature to copy an existing job that is similar to a new project, then modify the content.
GitLab provides a configuration file called ‘.gitlab-ci.yml’. It needs to be placed in the root of your repository and GitLab then automatically detects the file and run a job described in the file.
Jobs vs Pipeline + Jobs
In Jenkins, you will create a set of jobs (e.g. build, test and deploy) and connect them by triggering another job or use pipeline feature. For a typical API CI, we need build test and 4 deploy jobs (dev/qa/uat/prod) and glue one job to another.
GitLab provides a pipeline concept with ‘Stages’ (e.g. build, test and deploy). Each stage can have a multiple jobs and each job can have its own set of configuration including environmental variables or what branch to be executed.
Source Code Management vs The repository
Jenkins (still) supports CVS, Subversion and GIT and you can clone multiple repositories in one job. This is so generic and flexible you can fetch any number of repositories for build, test and deploy.
However, our CVS/Subversion were retired and migrated into GIT long time ago and this flexibility rather forces you to configure it every time.
You also need to somehow specify the branches to build, which does not work well when you follow Git Flow.
GitLab CI is available for each repository and you don’t need to tell any other system for how to fetch your repository.
Build Triggers vs When and Only/Except
In Jenkins, you need to specify how you want to fetch your code, either periodically or poll the repository.
If you want to run a job as soon as the code has been pushed for a branch, you need to frequently poll it or use Jenkins webhook and call it on a GIT repository.
GitLab CI pipeline runs automatically as soon as a change has been pushed for a branch. It also offers 3 simple and clear DSL, ‘When’ and ‘Only/Except’ to limit jobs running within the pipeline.
When: You can set manual to only trigger a job manually or make it dependent on another job using on_success/on_failure.
Only/Except can limit what branches or tags to run a particular job. We use this to ‘deploy production’ only on a master branch type of scenarios.
Build vs Stages
Jenkins Build supports a number of scripts (shell, batch, ssh) and Java build tool such as Ant/Maven by default (you can add more builds using plugins).
We use custom shell scripts to fetch some configuration items; run test; build a docker image; push to the Docker registry.
We could run pretty much the same script in build stage of the pipeline in GitLab CI. The scripts are identical and run on the same host machine and worked great.
At this stage, we encountered a problem. We need to initialise a test database prior to running a test. In Jenkins, we have a job for it and trigger it prior to the test job.
This database initialisation was only available on Jenkins so we had to call Jenkins webhook from a GitLab CI.
In GitLab CI, the only solution we found was to use curl to call Jenkins web hook trigger in ‘before_script’ section and ‘sleep for a period of time’ to finish the database initialisation. (You could make a better script and use Jenkins CLI/API to check whether a job has been completed).
Post-build Actions vs Stages and Triggers
In Jenkins, we use Post-build Actions to run a deploy job when the build job has been successfully completed.
In GitLab, you can create triggers to run another pipeline. A trigger can be either called from .gitlab-ci.yml (using curl) or from a webhook.
Jobs for each environment vs Pipeline
In Jenkins, we normally create a deployment job for each environment, development/QA/UAT/Production. This is due to the fact we need our client’s approval (UAT) prior to a release to a production and we cannot fully automate for a Continuous Delivery.
We use GitFlow to achieve the deployment process with a specific branch as follows:
- Development: Automatic deployment using develop branch
- QA (Internal): Nightly or on demand using develop branch
- UAT/Production: Manual using release/master branches
In Jenkins, it is not easy to make related jobs bundled and you need a very good naming conventions to make jobs somehow related each other. (Unless you use pipeline feature)
In GitLab CI, we can use stages and bundle jobs with DSLs such as environments/when/only/except so the pipeline is visualised clearly.
Lessons learned
We’re happy with our trial on GitLab CI and feel our CI processes are more seamless with our development activities thanks to a clear DSL with modern development practice.
We’ve since then created many CI processes using GitLab CI and there are a number of .gitlab-ci.yml templates we can reuse across projects.
The following are a few lessons learned on GitLab CI
- As of Jan 2020, Monorepo is not fully supported and we’re simply unable to use some Monorepo tool chain, which makes Jenkins little more appealing due to its flexibility
- Always create a project specific Runners instead of sharing by the capability of the Runner to avoid a conflict for Runner’s capabilities between projects
- GitLab is very active and GitLab Docs is comprehensive and easy to follow. Their issue tracker is also active and many issues are solved quickly, however there are bugs shipped from time to time and it might affect on your development activities when you are fully dependent on GitLab tool chain
- Late last year, GitLab shipped a critical bug in GitLab CI and we’re unable to build or deploy anything for a couple of days. We had to abandon all of Docker based builds for work around and wasted a lot of times
- ‘Multi-project pipeline’ is a feature currently only available for Premium users (we’ve been using a community free edition) and there is no other reliable way to visualise and manage multi-project pipelines (which makes hard to create a sophisticated pipelines for a complex project having many components)
