Allure Usage in Maven Projects

As a concise and easy-integrable reporting framework I mostly prefer the Allure report tool. Before giving a quick information and installation guide I’ll first explain why I chose this report tool and which language/frameworks I used with it.

★ Appearance is indeed cool and understandable within 7 pages shows the results in detail as below;

• Overview — includes overall report statistics and information
• Categories — shows defects by classification. There are two categories of defects by default as product defects (failed tests), test defects (broken tests). However, you can customize them by adding categories.json file to allure-results directory before report generation.
• Suites — structural view the results of your executed test cases.
• Graphs — includes charts demonstrating Status, Severity, Duration, Retries, Categories etc.
• Timeline — shows test execution time in sequential or parallel timing structure
• Behaviors — shows results aggregated according to stories and features.
• Packages — illustrates a tree-like layout of test results, grouped by different packages

★ Easy to install and integrate with our projects by following the similar steps which I demonstrated below.

★ Cheap to customize. You can show the logs or case/scenario steps on result rows.

★ History is readily accessible in the details of scenarios.

I work on the Maven framework where I use the Allure report. I’m going to begin with giving installation steps for Maven projects.

Installation

Selenium & TestNG — Maven Project Integration with Allure

1. Step: POM changes

a. Add Allure plugin and configuration

<plugin>
 <groupId>org.apache.maven.plugins</groupId>
 <artifactId>maven-surefire-plugin</artifactId>
 <version>2.20</version>
 <configuration>
  <testFailureIgnore>true</testFailureIgnore>
  <argLine>
   -javaagent:"${settings.localRepository}/org/aspectj/aspectjweaver/${aspectj.version}/aspectjweaver-${aspectj.version}.jar"
  </argLine>
  <suiteXmlFiles>
   <suiteXmlFile>testng.xml</suiteXmlFile>
  </suiteXmlFiles>
 </configuration>
 <dependencies>
  <dependency>
   <groupId>org.aspectj</groupId>
   <artifactId>aspectjweaver</artifactId>
   <version>${aspectj.version}</version>
  </dependency>
 </dependencies>
</plugin>

b. Add dependency

<dependency>
 <groupId>io.qameta.allure</groupId>
 <artifactId>allure-testng</artifactId>
 <version>2.13.0</version>
</dependency>

2. Step: Download Allure binary in keeping with your Allure version (you can find in POM.xml dependency as io.qameta.allure version)

a. If you use Windows, First move the downloaded allure file into your C directory or whatever you want to set path. Copy allure bin file path and set this path in Environment Variables > Path

b. On Mac you can run the following command to install Allure

brew install allure

3. Step: Check your Allure version on command prompt or terminal with the following command

allure --version

4. Step: Set your allure results’ directory. On Intellij you can create allure.properties to set directory (src/test/resources/properties/allure.properties)

allure.results.directory=target/allure-results

5. Step: Create TestNG.xml then run your tests through this xml

6. Step: Run below command on command prompt or terminal with the following command

allure serve

Selenium & Junit & Cucumber— Maven Project Integration with Allure

Same steps as above except 5th step. Instead of the 5th step you should create a TestRunner class like the instance below then run your tests through Junit.

@RunWith(Cucumber.class)
@CucumberOptions(features = { "src/test/resources/features" },
        glue = { "steps" },
        monochrome = true,
        plugin = {"rerun:target/failed_scenarios.txt",
                "io.qameta.allure.cucumber5jvm.AllureCucumberJvm"})
public class TestRunner extends AbstractTestNGCucumberTests {
    @DataProvider(parallel = true)
    @Override
    public Object[][] scenarios() {
        return super.scenarios();
    }
}

Appium & TestNG — Maven Project Integration with Allure

Similar steps in sequence; Addition in POM.xml, Allure binary installation, Create TestNG xml then run your tests through this xml, run allure serve command.

Rest Assured & TestNG — Maven Project Integration with Allure

Similar steps in sequence; Addition in POM.xml, Allure binary installation, Create allure.properties to set your results’ path (If it is not set Allure report is created in src by default), Create TestNG xml then run your tests through this xml, run allure serve command.

Allure Features

After installation you need to make the most of report features efficiently by using Allure annotations. You have opportunity to customize your results as you wish to see and understand with following annotations;

@Epic

@Features

@Stories/@Story

@Severity(SeverityLevel.BLOCKER)

@Description(“In this cool test we will check cool thing”)

@Step

@Attachment

@Link

• Epic, Story, Feature: In Agile development developers mostly use a tool to manage the tasks which includes user stories in product backlog and they give some labels to divide these tasks into smaller user stories. With the help of these annotations it’s easy to link our test method to a user story.

Usage in test class;

View in the report;

• Severity: By using this annotation you can define a value for your test to evaluate failed results easily. Allure has 5 levels of severity weight as BLOCKER, CRITICAL, NORMAL, MINOR, TRIVIAL.

Usage in test class;

View in the report;

• Description: In order to make the test case more understandable this annotation can be used to give an expressive statement.

Step: This annotation is generally used in BDD automation projects as a statement defined test step accurately. If you use Cucumber, Gauge or these kind of behavior driven development frameworks, you should define Step tags in your step or page classes.

Usage in step/test class;

View in the report;

• Attachment: this annotation returns your added attachments as String or byte. In my opinion the best feature of the Allure is this annotation. Because you can add whatever you want to highlight or show like screenshots for UI test cases or your call detail for backend test cases. Firstly you should create a method that demonstrates what you want to show in the report. Then call this method where you want to utilize it (it can be put in the After part of your test or in a more specific condition such as failure). I’ll show the way to use the attachment part for a Rest Assured project.

1- Add following dependency:

<dependency>
   <groupId>io.qameta.allure</groupId>
   <artifactId>allure-rest-assured</artifactId>
   <version>2.13.0</version>
</dependency>

2- Add following code part where you give your Request Specification

.filter(new AllureRestAssured())

View in the report;

• Link: You can put any link you want to direct from your test execution result such as Bug tracking panel or confluence page related business analysis. Manipulation pattern of your link is possible by adding custom path into the system property in the following format:

allure.link.mylink.pattern=https://restful-booker.herokuapp.com/{}

Allure will replace {} placeholders with the value specified in the annotation.

Jenkins Integration

Lastly, the Jenkins integration is a necessity for SDLC. By adding a plugin and configuration of this added plugin you can easily see your Allure report on the Jenkins build page.

1- Go to Jenkins -> Manage Jenkins -> Manage Plugins

2- Search for Allure plugin and install.

You will see installation process;

3- Go back to the Manage Plugins page and validate installed Allure Jenkins Plugin in Installed tab.

4- Go to Jenkins -> Manage Jenkins -> Global Tool Configuration

5- Configure Allure Commandline which is located on the bottom of the Global Tool Configuration page.

6- Go to your job’s configuration page (job/{your job service}/configure). Add post-build action into your job’s Build Settings.

7- Build your job and see the Allure report.

In brief we learnt the usage and benefits, installation and Jenkins integration of the Allure report tool. In order not to repeat the same things I’ll suggest one last thing: don’t give up while trying to add a report tool into your project. Although adding a report tool steps seems like a piece of cake ,there may occur some specific errors about your versions or structure and I know this is a pain in the neck! So follow general logic about integration of external libraries or tools. Alternatively you can find many sample projects in Github or in a similar platform then you can imitate the way of adding a report tool.

I hope you find this article helpful. Please do not hesitate to contribute or contact me :)

My Sample Github Project:

https://github.com/fsebalien/hotelbookingBE

References:

https://docs.qameta.io/allure

https://www.seleniumeasy.com/selenium-tutorials/allure-report-example-with-annotations

https://www.swtestacademy.com/allure-report-testng/