Reporting test results in GitLab

Hi all! In this story, I will be telling about how to report test results with jUnit and HTML based (etc. Allure) in GitLab and GitLab pages after running pipeline.

Let’s see what the reports will look like before start…

jUnit Report

Test report displaying

Viewing details of a failed test

Click here to watch the video recording of jUnit report

HTML based report (used Allure for this story)

Displaying index.html

Click here to watch the video recording of HTML report

Let’s Report!

jUnit Reporting
Edit your .gitlab-ci.yml as follows

image: maven:3.6.3-jdk-11

stages:
  - Test

Test:
  stage: Test
  script:
    - mvn clean install
  artifacts:
    when: always
    paths:
      - target/JUnitReport/
    expire_in: 1 week
    reports:
      junit: target/JUnitReport/report.xml
  only:
    - master

for this pipeline script, after test script ran, JUnit report is generated at target/JUnitReport/ path.

Artifacts filed provides to store files in GitLab.
1) You need to define JUnit path where the JUnit report is generated to artifacts > paths as follows. It stores files in the path you defined to gitlab. This is the path that stores the .xml file generated by JUnit.
2) reports > junit field provides to display JUnit report in GitLab ci job. So, it is needed to define the path of the .xml file generated by JUnit under the path we defined at step 1.
3) expire_in field provides to store reports until the time you define. In this example, JUnit artifacts will be deleted after 1 week.

artifacts:
    when: always
    paths:
      - target/JUnitReport/
    expire_in: 1 week
    reports:
      junit: target/JUnitReport/report.xml

After the pipeline run, you can display test results in your pipeline.
To see the results, follow the steps; project in your GitLab > CI/CD > Pipelines > Click on pipeline number > Click Tests tab

HTML based Reporting
Edit your .gitlab-ci.yml as below; We have 2 stages in this pipeline (Test and Report).

Test:
  stage: Test
  script:
    - mvn clean install
    - mvn allure:report
  artifacts:
    when: always
    paths:
      - target/allure-report
    expire_in: 1 week
  only:
    - master

Report:
  stage: Report
  only:
    - master
  script:
    - mkdir public
    - mv target/allure-report/* public
  artifacts:
    when: always
    paths:
      - public
    expire_in: 1 week

Test stage runs tests then exports allure test results as HTML to target/allure-repot path. And then stores target/allure-repot path until next week.

Test:
  stage: Test
  script:
    - mvn clean install
    - mvn allure:report
  artifacts:
    when: always
    paths:
      - target/allure-report
    expire_in: 1 week
  only:
    - master

Now! We have stored our report files.
The Report stage moves the files which are stored by the Test stage to public folder.

Report:
  stage: Report
  only:
    - master
  script:
    - mkdir public
    - mv target/allure-report/* public
  artifacts:
    when: always
    paths:
      - public
    expire_in: 1 week

After this pipeline ran, you can display test results in your pipeline.
To see the results, follow steps; project in your GitLab > CI/CD > Pipelines > Click on pipeline number >Click Jobs tab > Click Job Id under Report stage > Click Browse button under Job artifacts

Click public path then click HTML file which you want to display. The report will be displayed.

Also, you can send URL of the HTML report to a slack channel. I have report.sh file that can send a slack message. report.sh file takes parameters of Project name, Job id, and base GitLab pages URL of your project.
These variables are provided by gitlab;
${CI_PROJECT_NAME} : Project Name
${CI_JOB_ID} : Job Id
${CI_PAGES_URL} : projects gitlab pages url

Report:
  stage: Report
  only:
    - master
  script:
    - mkdir public
    - mv target/allure-report/* public
    - sh .report/slack.sh ${CI_PROJECT_NAME} ${CI_JOB_ID} ${CI_PAGES_URL}
  artifacts:
    when: always
    paths:
      - public
    expire_in: 1 week

report.sh

#!/bin/sh

PROJECT_NAME=$1
JOB_ID=$2
PAGES_URL=$3

find=".com"
replaceWith=".com\/-"
newUrl=$(echo "$PAGES_URL" | sed "s/$find/$replaceWith/")
newUrl="$newUrl/-/jobs/$JOB_ID/artifacts/public/index.html"

message="$PROJECT_NAME results $newUrl"
curl --location --request POST 'https://hooks.slack.com/services/..........' \
--header 'Content-Type: application/json' \
--data-raw "{ \"text\": \"$message\", \"channel\": \"ugc-test-alert\"}"

After pipeline ran slack message with report URL will be sent

Bye! :)