How to integrate Allure reports for automated tests in Jenkins X

Dailymotion gets higher involvement of its feature teams in Quality by getting faster and better feedback on the execution of tests in a CI/CD pipeline

Maxime BELLOU
Dec 20, 2019 · 5 min read

As part of Dailymotion’s DevOps initiatives, we decided to move from Jenkins to Jenkins X to be faster in our release cycle and to further empower our teams. This article talks about how we migrated our Allure test reports using Jenkins X.


To accelerate our release process we started adopting CI/CD best practices, one of which was to implement Jenkins X and Kubernetes in our CI pipeline. We believe another key value is sharing information across different teams. To share and communicate about the result and coverage of our automated tests, we decided to use Allure, as it is a tool that produces great reports from your automation codebase. The challenge was to plug Allure with Jenkins X.

Example of a report in a Java project

One of our UI component integration tests is using Java, Maven, Selenium and TestNG. We decided to add an extra layer with Allure to produce nice-looking reports.

There’s no reason to reinvent the wheel when it comes to integrating Allure into your project. Just follow the instructions from the documentation. Allure has a plugin that integrates well with Maven and TestNG, so that when you execute your Maven command mvn test, Allure will automatically produce some files to build a report.

As we were using Jenkins, we wanted the report to be directly integrated into the Jenkins build. The Allure plugin also makes this part very easy — simply follow the guide to install it on Jenkins, let the magic happen, and appreciate your first nice-looking report!

OK, now it’s time to take a look at the serious stuff and move to Jenkins X. Our current setup is using “Static Jenkins” in Jenkins X and our pipelines are defined using the Jenkins Declarative Pipeline DSL. A pipeline defines what stages and steps will be executed during the build. One step of our pipeline is to execute our test in a Maven container. We execute the Allure plugin in a post step of the stage. Here is an example of a Jenkinsfile:

Example of Jenkinsfile with Allure

This is the theory, but in practice, some complications arose and made it more complex.

The first issue we faced was installing the Allure plugin on our Jenkins X server. Installing the plugin manually through Jenkins UI is easy, but as mentioned in our previous article in the section: the Gitops applied to Jenkins X, the Helm chart used to install Jenkins has a startup script that resets the Jenkins configuration with values coming from a Kubernetes ConfigMap. You’ll need to update this ConfigMap by adding the Allure plugin command-line installation XML. This way, when the startup script runs, it will always reinstall the Allure plugin.

If you manually install the Allure plugin on your Jenkins pod, you can then rsh in your pod ( jx rsh <jenkins-pod-name> -n <jenkinsx-namespace>) and find the XML file for the Allure plugin in /var/jenkins_config/. You’ll need to add the content of this XML file in your ConfigMap used by Helm chart to install Jenkins.

It will look like this:

Extract of the ConfigMap XML file for Helm

We now need to override the configuration for the plugin in the apply_config.sh part of the ConfigMap:

apply_config.sh: |-
rm -rf /var/jenkins_home/ru.yandex.qatools.allure.jenkins.tools.AllureCommandlineInstallation.xml
ln -s /var/jenkins_config/ru.yandex.qatools.allure.jenkins.tools.AllureCommandlineInstallation.xml /var/jenkins_home/ru.yandex.qatools.allure.jenkins.tools.AllureCommandlineInstallation.xml

This means that the plugin will always be installed when the startup script will be executed. The second issue we faced was the file system permissions on the Allure files. When the step in the Integration tests stage is executed in the Maven container, the default user is root. It means that all the result files generated by Allure were created by the root user.

The Allure script to generate the report is executed in the default container jnlp. The default user for this container is jenkins. As the containers in the pod share the same volumes, when the Allure script tries to write the Allure report files, it gets an access denied error. To solve this, we had to change the default user of the Maven container to jenkins user. To achieve this, we need to add the following parameters to the container description:

securityContext:
runAsUser: 1000

Where 1000 is the UID of the jenkins user in the container.

Here is how the final Jenkinsfile looks like:

Final Jenkinsfile with Allure integration

The test report can be found in the Jenkins build (in classic view, Blue Ocean is not yet supported):

Allure will take care of marking the Jenkins build as unstable if there are test failures.

Here is where you can find a demo report to see how an Allure report looks like: https://demo.qameta.io/allure/

Now that we have integrated this reporting tool in a Java project built in Jenkins X, the goal is to extend it to all our projects. They use different technologies and languages. Some languages are supported by Allure, like Python or JS, but some, such as Golang, are not. We started to develop an open-source library that allows us to integrate Allure in our code and generate Allure results for Go projects. Here is the link: https://github.com/dailymotion/allure-go


Having a nice-looking report is not the only benefit we get from Allure. It allows us to establish better communication between the testers, developers and Product Owners. In a TDD approach, it allows the Product Owner to directly review the test script logic without having to know how to read code.

Our testers are involved in the design and development phases in an earlier stage, the developers have better visibility on the tests and are much more involved in Quality. It also helps to investigate quicker, as we don’t need to search for an error in hundreds of lines of logs. The steps are directly highlighting the errors and pasting the error stack trace in the report. We also use its features and stats to get rid of our test documentation and to estimate the coverage of our functional tests.

Dailymotion

The home for videos that matter

Thanks to Vincent Behar, Dailymotion Team, Alice Girard Guittard, Rachel Wignall, and Zala Vogrinc

Maxime BELLOU

Written by

Dailymotion

The home for videos that matter

More From Medium

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade