5. Allure Report

Okay guys. We design the test strategy, create a structure and implement the tests for our web application. Please see the blogs that I published before.

1. Building Hybrid Test Automation Framework - Python

2. Selenium Webdriver

3. Pytest Test Framework

4. Page Object Model Pattern

What else? It is time to deliver a test report to all the stakeholders who are interested in the quality of the application. After the tests are completed, the test report should be generated. The test automation report should be as clear as possible so that to be understood by everyone. In this post, I will tell you about an HTML Test Report Tool, Allure Report. It has amazing features, and easy to get into immediately. Here is what will we see together in this post. Happy reading!

1. What is Allure Report?
2. Installation of Allure Report
3. Using Allure Report

1. What is Allure Report?

Allure Report tool is a flexible lightweight test report tool. It allows everyone participating in the development process to extract the maximum of useful information from everyday execution of tests. Allure Report provides a nice big picture of which features have been covered, where defects are clustered, how the time of execution takes and so on for the managers as well. Allure report is generated in two steps. First of all, during test execution, “adapter” library that is attached to the framework called and saves information of test execution to XML files. Java, Python, C# are some popular examples for the adapters. After the test execution is completed, XML files are transformed to an HTML report by “report generation”. This can be done with the command line tool.

2. Installation of Allure Report

To generate test automation report you only need to complete just a few simple steps;

1. Downloading and installing Allure command-line tool suitable for your environment.
2. Locating test execution data that you have to build a report on.

Since we will Allure Report with Python and Pytest, I will show you how to install Allure with these. If you don’t prefer to use Pytest on your test automation structure, you can find the other allure adapters in this link. Although this link seems old, the adapters still work fine on the list. So, we are ready to install Allure via pip install in command line. First of all, open your command window and type this.

$ pip install allure-pytest

If you want to verify your installation is done without any error, type

allure --versioncommand and see the result below. My allure adapter version is 2.9.0 as you can see. That’s it! You are ready for a fancy test automation report :)

Verify the allure report installation

3. Using Allure Report

After a successful installation, we can decorate our test_logincode in POM tutorial with Allure features. Here is the test_login.py file that extended allure features. There are three decorators related to BDD in our code:@allure.epic , @allure.feature and @allure.story to mark your tests according to Epic/Feature/Story breakdown. Also, to mark your tests by severity level you can use @allure.severity decorator. It takes a allure.severity_level enum value as an argument. To make a connection with your report and a test management system Allure has @allure.link, @allure.issue and @allure.testcase descriptors. Reports should have a screenshot if a problem occurs once the test failed to display many different types of provided attachments that can complement a test, step or fixture result.

test_login.py

It is time to run the test, gather results and generate a test report. To enable Allure listener to collect results during the test execution simply add

--alluredir option and provide a path to the folder where results should be stored seen below.

$ py.test --alluredir=%allure_result_folder% ./test_login.py

allure_result_folder is your location to be stored in the test results as an “.XML” files. To see the actual report after your tests have finished, you need to use Allure command-line utility to generate a report from the results.

$ allure serve %allure_result_folder%

This command will show you generated a report in your default browser. Here is some screenshot in my application.

Overview Page

Suites Page

Graphs Page

That’s all about Allure Report so far. The next step is integrating our test automation codes into Jenkins. See you there :)

6. Jenkins and Allure Report Integration