Behave-allure integration (English)

Note: This text is still being done as the development and the configuration of allure that I have done be in a study, any discovery and learning about this will be placed in there.

Report:

The allure report shows the data according to the execution, that is if some feature has a scenario outline each execution will be one suite showed on allure, and also each row of the feature scenario outline generates a JSON report file.

Categories:

The categories are divided in another JSON file called “categories.json” in the report folder together with report JSON generated by allure, that makes one chain/match with the information that been generated by behave, he can perform filters of information and separates them into categories(suggest: create one file with all possible categories and use the same as pattern, because all the possibilities they can give in the tests will be saved and shown in final report without having to change them).

[
{
"name": "Ignored tests",
"matchedStatuses": ["skipped"]
},
{
"name": "Test defects",
"matchedStatuses": ["broken"]
},
{
"name": "Product defects",
"matchedStatuses": ["failed"]
},
{
"name": "Passed tests",
"matchedStatuses": ["passed"]
}
]

Also are possible define exceptions to categories, for example, this JSON part that can be added, that if there is the exception of “not found”, can be separated in a category:

{
"name": "Element not found",
"traceRegex": ".*NoSuchElementError.*",
"matchedStatuses": ["failed"]
}

Environment:

There are two ways to show the environment that was been executed the tests in the allure report, both are easy to configure, the first, easier, its a file called “environment.poperties” in which you can put the environment as you wish, example:

browser=Google chrome

chrome_version=78.0.3904.87 64 bits

environment=Feature

environment_link=http://feature.com/#/login

The second way would be a file called “environment.xml”, that being necessary to carry out the configurations above using an XML file, that works the same, for example:

<?xml version=”1.0" encoding=”UTF-8"?>
<environment>
<parameter>
<key>browser</key>
<value>Google Chrome</value>
</parameter>
<parameter>
<key>chrome version</key>
<value>78.0.3904.87 64 bits</value>
</parameter>
<parameter>
<key>environment</key>
<value>Feature</value>
</parameter>
<parameter>
<key>environment link</key>
<value>http://feature.com/#/login</value>
</parameter>
</environment>

Both the files are located in the reports folder of allure (just one required to work)

Screenshots:

it’s possible to add screenshots on steps with that code:

from allure_commons._allure import attach
from allure_commons.types import AttachmentType
import allure
allure.attach(context.browser.get_screenshot_as_png(), name='Screenshot', attachment_type=AttachmentType.PNG)

The definition at what time use screenshots to improve the test report goes on your way if you want a screenshot after all steps append a function “after_step” in your environment.py file so that the screenshot is taken at each step. if you want only after an error in step, just put an “if” as in the following example:

def after_step(context, step):    
    if step.status == "failed":      
        allure.attach(context.browser.get_screenshot_as_png(), name='Screenshot', attachment_type=AttachmentType.PNG)

And finally, if you want to put in a specific step, just put the code on your step, without forgetting the imports.

Trending:

The history trend mainly serves to shows in graphs the features was been executed in that instant, so how it’s a new feature or a new execution of an old, the time that they took for being executed and they categories.

The history trend, despite saving all JSON files in that has been defined folder, will show only the last twenty executions.

I divided the commands for the execution of the history into three steps, first, second and third execution, same the second and the third execution have few differences has an important step that should be considered.
it is also worth mentioning that the third execution can be repeated for all future executions.

• First execution:

behave -f allure_behave.formatter:AllureFormatter -o allure/results ./features

That first command is defined the formatter and the report JSON output folder, if in your “behave.ini” is defined the formatter and the output folder, just execute the command “behave ./features”. it’s interesting to let it be withdrawn in the “behave.ini” file so that the report can be executed in any computer, up until the one that is not with the file.

allure generate allure/results/ -o allure/reports

That second command generates the allure report folder, and append the first execution in the files of the “history” folder.

so enough open the allure report with the command:

allure open allure/reports

Note: aways be good define the categories and environment files at first, since these files normally don’t be changed routinely and serve as an important font of information and standardization of report.

• Second execution:

Repeat the execution command:

behave -f allure_behave.formatter:AllureFormatter -o allure/results ./features

And then we must copy the history folder generated for the results folder, to generate a new report with the following command:

cp -R allure/reports/history allure/results/history

But why it important the folder is copied?

When the first report of the first execution was generated, he takes the report and generated on the allure folder, now we have new reports, that is, with this copy we are appending the history of the old execution to the new execution, thus creating its execution history, without that command he will see that new execution as the first, what is not desired.

So let’s generate this new execution report:

allure generate allure/results/ -o allure/reports --clean

And open the report:

allure open allure/reports

It’s possible to see the difference, both in the initial screen and in the graphs screen.

• Third execution:

We execute the behave:

behave -f allure_behave.formatter:AllureFormatter -o allure/results ./features

There is the difference in the second execution, before copy the history folder from report to results, we must delete the old history folder. The folder can be deleted manually without problems, but how am I running the commands in PowerShell with VSCode I just ran the command:

Remove-Item .\allure\results\history\ -Recurse

Only then we can copy the history folder:

cp -R allure/reports/history allure/results/history

Generate the third execution report:

allure generate allure/results/ -o allure/reports --clean

And open the report:

allure open allure/reports

From here, just repeat the steps of the third run and the Allure history will work perfectly.

Also are important to quote the Allure reports portal, in which the reports of several executions can be added, but I will not go into many details at the moment.

Directory example after the third execution:

C:.
├───.vscode
├───allure
│   ├───reports
│   │   ├───data
│   │   ├───export
│   │   ├───history
│   │   ├───plugins
│   │   └───widgets
│   └───results
│       └───history
├───features
│   ├───profiles
│   ├───steps
│   └───TC
└───tools

Thanks:

This text was only possible thanks to the team that I belong to, Rebecca as my leader introduced me to the world of QA and encouraged me to write this text, Jeckson helped me in the definition of how the allure worked and in the incessant research on the internet about its functioning as well as helping to encourage this to be written, Karol was the discoverer of how to automate allure itself, and Gabriel with support and explanations. Without this team, this text would not exist nor would I know anything about this subject, thank you!