How to integrate Allure reports for automated tests in Jenkins X

Dailymotion gets higher involvement of its feature teams in Quality by getting faster and better feedback on the execution of tests in a CI/CD pipeline

Dec 20, 2019 · 5 min read
Image for post
Image for post

As part of Dailymotion’s DevOps initiatives, we decided to move from Jenkins to Jenkins X to be faster in our release cycle and to further empower our teams. This article talks about how we migrated our Allure test reports using Jenkins X.

To accelerate our release process we started adopting CI/CD best practices, one of which was to implement Jenkins X and Kubernetes in our CI pipeline. We believe another key value is sharing information across different teams. To share and communicate about the result and coverage of our automated tests, we decided to use Allure, as it is a tool that produces great reports from your automation codebase. The challenge was to plug Allure with Jenkins X.

Integrating Allure in a testing project to generate great reports

Image for post
Image for post
Example of a report in a Java project

One of our UI component integration tests is using Java, Maven, Selenium and TestNG. We decided to add an extra layer with Allure to produce nice-looking reports.

There’s no reason to reinvent the wheel when it comes to integrating Allure into your project. Just follow the instructions from the documentation. Allure has a plugin that integrates well with Maven and TestNG, so that when you execute your Maven command mvn test, Allure will automatically produce some files to build a report.

As we were using Jenkins, we wanted the report to be directly integrated into the Jenkins build. The Allure plugin also makes this part very easy — simply follow the guide to install it on Jenkins, let the magic happen, and appreciate your first nice-looking report!

OK, now it’s time to take a look at the serious stuff and move to Jenkins X. Our current setup is using “Static Jenkins” in Jenkins X and our pipelines are defined using the Jenkins Declarative Pipeline DSL. A pipeline defines what stages and steps will be executed during the build. One step of our pipeline is to execute our test in a Maven container. We execute the Allure plugin in a post step of the stage. Here is an example of a Jenkinsfile:

Example of Jenkinsfile with Allure

This is the theory, but in practice, some complications arose and made it more complex.

Issues we faced (plugin installation, security context…)

The first issue we faced was installing the Allure plugin on our Jenkins X server. Installing the plugin manually through Jenkins UI is easy, but as mentioned in our previous article in the section: the Gitops applied to Jenkins X, the Helm chart used to install Jenkins has a startup script that resets the Jenkins configuration with values coming from a Kubernetes ConfigMap. You’ll need to update this ConfigMap by adding the Allure plugin command-line installation XML. This way, when the startup script runs, it will always reinstall the Allure plugin.

If you manually install the Allure plugin on your Jenkins pod, you can then rsh in your pod ( jx rsh <jenkins-pod-name> -n <jenkinsx-namespace>) and find the XML file for the Allure plugin in /var/jenkins_config/. You’ll need to add the content of this XML file in your ConfigMap used by Helm chart to install Jenkins.

It will look like this:

Extract of the ConfigMap XML file for Helm

We now need to override the configuration for the plugin in the part of the ConfigMap: |-
rm -rf /var/jenkins_home/
ln -s /var/jenkins_config/ /var/jenkins_home/

This means that the plugin will always be installed when the startup script will be executed. The second issue we faced was the file system permissions on the Allure files. When the step in the Integration tests stage is executed in the Maven container, the default user is root. It means that all the result files generated by Allure were created by the root user.

The Allure script to generate the report is executed in the default container jnlp. The default user for this container is jenkins. As the containers in the pod share the same volumes, when the Allure script tries to write the Allure report files, it gets an access denied error. To solve this, we had to change the default user of the Maven container to jenkins user. To achieve this, we need to add the following parameters to the container description:

runAsUser: 1000

Where 1000 is the UID of the jenkins user in the container.

Here is how the final Jenkinsfile looks like:

Final Jenkinsfile with Allure integration

The test report can be found in the Jenkins build (in classic view, Blue Ocean is not yet supported):

Image for post
Image for post

Allure will take care of marking the Jenkins build as unstable if there are test failures.

Here is where you can find a demo report to see how an Allure report looks like:

Extending to all projects

Now that we have integrated this reporting tool in a Java project built in Jenkins X, the goal is to extend it to all our projects. They use different technologies and languages. Some languages are supported by Allure, like Python or JS, but some, such as Golang, are not. We started to develop an open-source library that allows us to integrate Allure in our code and generate Allure results for Go projects. Here is the link:

Having a nice-looking report is not the only benefit we get from Allure. It allows us to establish better communication between the testers, developers and Product Owners. In a TDD approach, it allows the Product Owner to directly review the test script logic without having to know how to read code.

Our testers are involved in the design and development phases in an earlier stage, the developers have better visibility on the tests and are much more involved in Quality. It also helps to investigate quicker, as we don’t need to search for an error in hundreds of lines of logs. The steps are directly highlighting the errors and pasting the error stack trace in the report. We also use its features and stats to get rid of our test documentation and to estimate the coverage of our functional tests.


The home for videos that matter

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store