Automating the Automation Server (Jenkins)

Continuous Integration (CI) is the idea of continuously integrating developers’ work in a shared mainline, i.e. daily or multiple times a day. The process relies on having automatic tests continuously providing feedback about the health of the code base and it aims to help developers deliver faster and more reliably.

At Feedzai, we continuously try to upgrade our CI processes to help developers code faster and with even more confidence in the quality of their code. An important feature of our CI process is that the mainline Source Code Management branch is stable at all times. We ensure this by requiring feature branches to go through the CI pipeline before entering the mainline branch and going through the CI pipeline a second time.

A considerable amount of time is dedicated to designing, implementing and automating the CI processes that we want to enforce. With that in mind, one of the most recent efforts we are making is automating the usage and configuration of the CI platform we use the most, Jenkins. With this automation we expect to improve our reaction time to requirement changes in CI infrastructure configuration management as well as improve and easily plug in custom code in the scripts we use to build our Software.

We are currently in the process of migrating our Jenkins configurations to code, using both the Job DSL plugin and the Pipeline plugin. Job DSL provides a way to, using Groovy scripts, create and configure Jenkins jobs. Jenkins pipelines provide a programmatic interface to script the orchestration of the multiple build steps needed to build the Software, also using Groovy, as well as to access to a bash shell in the pipeline code, among other Jenkins plugins that have pipeline bindings. Together, the two plugins provide a solid foundation for having our project’s CI configuration fully integrated in the project’s repositories in the form of Groovy code.

The idea of using this stack to achieve this is not new. At first, having a Jenkins job configured programmatically was achievable only with the Job DSL plugin, which allowed for the complete configuration of any type of job in Jenkins. However, since the Pipeline plugin came along, the community support and development seems to have shifted towards this approach, and that is the reason why we chose to go this way.

What we are doing right now, and what I would like to talk to you about, is a shared library to help our projects making the most of these plugins. Our projects have similarities both in their processes and their infrastructure. The goal of this library is to leverage these common paths and provide them behind an easy-to-use API that can be then consumed in both pipeline and job DSL scripts. For example, since most projects rely on the same platform for code reviews, this library integrates pipeline jobs with the code review platform out of the box.

Introducing ‘Magritte’

Inspired in the surrealist painting that reads “ceci n’est pas une pipe”, authored by the french painter of the same name, René Magritte, this is the name of the shared library we have been implementing. The name comes from the idea that, even though some of the code in Magritte looks like a pipeline, it is not, since the concrete implementation of the steps that actually build the software, i.e. the pipeline, is defined in the pipeline scripts implemented by each project.

Now that we have a name and picked up a logo, it is time to tell you what surrealism looks like at Feedzai. Well, not actually surrealism, nor any kind of art, but the shared library we’ve been talking about. Currently Magritte has 2 modules:

1. A Job DSL library available for projects in their Job DSL scripts where jobs are created and configured;
2. A Pipeline Shared Library that can be used by projects in their pipeline scripts to help orchestrating the build steps.

With the Job DSL library, Magritte provides a simplistic way for projects to create and configure new jobs, namely pipeline and seed jobs since these are in fact the major kinds of jobs that we want to have in the long run. The beauty of this library is that, if we were to integrate our jobs with a new service or plugin in the future, all we would need to do is a release of Magritte with the new features in place and run the seed jobs that generate the configuration in Jenkins. Below is in an example of a pipeline job being created in a Job DSL script with this library. This snippet is actually being used to create the job that runs Magritte’s unit tests in CI.

PipelineJobFactory.builder()
        .inJobDSLContext(this)
        .keepAtMostBuilds(15)
        .withProject(“feedzai/magritte”)
        .withJobName(“develop-1.X-unit-tests”)
        .withJobDescription(“Run Magritte unit tests”)
        .withProjectBranch(“develop-1.X”)
        .withJenkinsFile(“jenkins-ci/pipelines/unit-tests.groovy”)
        .build()
        .createJob()

Our projects are also able to use Magritte in their pipelines scripts, using the pipeline shared library module in Magritte. The idea is that ,by using this library, the projects are freed from having to implement logic for ensuring a clean environment to execute tests and other common tasks such as reporting the build results in the end. Ideally, the pipeline scripts only have the logic needed to build and test the project, while everything else is hidden inside the pipeline shared library module in Magritte. Below is an example of the pipeline that runs Magritte’s unit tests in CI. Even though the pipeline looks short, it actually has failure handling logic and an environment setup stage that are hidden behind the shared library.

@Library(“Magritte@develop-1.X”)
import com.feedzai.jenkins.common.FileSizeUnit
import com.feedzai.jenkins.flow.StandardBuildFlow
import java.util.concurrent.TimeUnit
final StandardBuildFlow buildFlow = StandardBuildFlow.builder()
    .withExecutorLabel(“general-linux”)
    .withAbsoluteTimeout(15, TimeUnit.MINUTES)
    .withFrozenLogTimeout(5, TimeUnit.MINUTES) // Abort when stuck
    .withLogFileSizeLimit(15, FileSizeUnit.MEBIBYTE)
    .withPostBuildSteps {
        junit ‘**/TEST*.xml’
    }
    .build()
buildFlow.execute(this) {
    stage(“Clone”) {
        fdz_cloneFdzRepo(“feedzai/magritte”, params.FDZ_GIT_ID)
    }
    stage(“Test”) {
        fdz_invokeMaven([‘clean’, ‘compile’, ‘test’, ‘-Pjacoco’])
   }
}

Magritte is a fully fledged Software project. It has code, tests, quality metrics, documentation and a release cycle. We use Groovy to develop, Maven to build, Junit and Surefire to code and run our tests, Jacoco and Sonar to have metrics about the code and GroovyDoc and Gitbook to document all the APIs and features of the project. It also has its own CI pipelines and jobs (that also use Magritte) that can be run to test the code and release/deploy the project.

Currently, Magritte is still on an very early stage, being used only by a few of our projects, including itself. However, we are hopeful that this project will help us improving the life of Developers and Quality engineers once it becomes more widespread as it provides:

1. A project where new automations can be easily added for testing new flows and improvements to existing automations (extensibility);
2. A way for quickly updating the CI configuration of all projects using Magritte (agility);
3. A streamlined way of executing builds in our CI platform (consistency);
4. An easy to use API for defining jobs and build/test our platform (usability).

Hopefully, in the near future, Magritte will be deployed in most of our projects and these improvements will become more and more tangible. For example, we want to set up our CI workers as an internally hosted Cloud in the future. If by that time we have a widespread usage of Magritte, all we will need to do is code, test, commit and push the new feature. It’s that simple!

At Feedzai, we love automating stuff. It can be a cumbersome task, but with the right tools in place, we want to make it easy to create automations that can be used across the entire company in the blink of an eye. Magritte is part of the car that will take us in the journey of automating everything at Feedzai. That’s right, the car! Did you think we were going to walk? We’d rather automate stuff. If you love automation, you can automatically join us at Feedzai, where you will face awesome challenges alongside an amazing team. If you want to be part of it, we’ve got good news for you: We’re hiring!