Integrating Bug Tracking with Your Test Automation Framework
You ran your test and noticed it failed. The first logical step is to open your issue tracking tool and create a bug ticket by manually entering the details of the failure. Wouldn’t it be much easier though, if you could create a ticket right from your test report every time a test fails? In this article, we will demonstrate how this can be done with a single click.
If we want to call our testing workflow efficient and successful, it is essential to have test reports generated once the test execution is complete. Learning how and why our tests failed is important to avoid future errors and improve the quality of our tests.
Creating a bug ticket for each failed test case is another important component of an effective testing workflow. It helps us to easily track and resolve all the issues and bugs we may encounter in our tests.
How is it usually done?
After test execution is finished, many testing tools generate some kind of test report to show us all important details regarding our test cases, i.e. if they failed, where they failed, the reasons why they failed, etc.
So, we take all those details and then manually create a bug ticket in our issue tracking tool.
While this is the common approach, it’s not the most convenient. Most of these reports are generated in the XML format, which makes things a bit more complicated. It can be hard to decipher the real reason why the test failed when you’re reviewing a chunk of code that isn’t always easy to read.
It would be much better if we could create bug tickets automatically from our test reports when the test case fails. This can be accomplished by integrating our test results with a third-party issue tracking or project management tools such as Atlassian Jira, Azure DevOps, or Bugzilla right in the testing environment. This gives us the option to create a bug ticket in case there is a failed test in the build.
Direct integration of a bug tracker into the testing environment means significantly less manual work since the reporting of detected problems in the testing environment is fully automated. This also implies that there would be an improvement in the productivity and quality of our testing process.
The question is, which issue tracking tool should we choose for this task? Well, it all depends on your preferences and needs, but the obvious choice for many is Atlassian Jira.
Atlassian Jira is the most popular issue tracking and management system among developers and companies worldwide. Jira’s capacity to manage issues, plan sprints, distribute tasks, customize workflow, track code, and production changes, as well as the ability to support test case management is what makes this tool essential for modern DevOps teams. Jira can also be easily integrated with other software and tools, including those used for test automation.
We’ll demonstrate how simple it is to connect your Jira project to a test automation tool and create bug tickets directly out of your test reports.
The test automation tool we’ll use in this example is Ranorex Webtestit, a complete web testing toolset built to speed up and simplify testing with Selenium and Protractor.
Ranorex Webtestit does this by handling the tedious test setup process for you by scaffolding the test framework, setting up required drivers, and creating boilerplate code, saving you precious time that could be used for testing.
Ranorex Webtestit also uses the Page Object pattern to structure your test code, allowing you to easily generate selectors and send them to the created Page Objects with the help of the free Ranorex Selocity extension. The extension enables you to initialize actions by drag and drop and makes it easy to set up endpoints for running tests against different browsers, operating systems, and devices.
Last but not least, Ranorex Webtestit generates reports during each test execution. These reports allow you not only to view all imported details related to your executed test cases but also to automatically create issues for failed test cases in a Jira project and resolve issues after re-testing them.
To do this, we need to connect Ranorex Webtestit to our Jira project, which requires downloading and installing Ranorex Webtestit and having a JIRA account.
To start, open Ranorex Webtestit and create a test project. Here are the tutorials on how to create a new project and how to write your first test in Ranorex Webtestit.
Let’s have a look at the first steps of JIRA integration. In Ranorex Webtestit, open Preferences > Project Settings > Apps & integrations and fill out all required fields.
The parameters in ‘Completed status name’ and ‘Active status name’ fields must match the values set for complete and active status in your Jira project in order to allow you to also reopen issues right from your test reports.
When you click the ‘Connect’ button, you’ll be prompted to enter your Jira username and password.
If you use Jira Server edition, enter your Jira username in the ‘Username’ field. If you use Jira Cloud edition, enter your dedicated email address for the username as explained in this Atlassian Community post.
Required parameters for the ‘Password’ field also vary depending on whether you are using Jira Server or Cloud edition. The Server edition requires your Jira account password, while Jira Cloud edition requires an API Token to be created.
To create an API Token, do the following:
- Log in to https://id.atlassian.com/manage/api-tokens
- Click ‘Create an API token.’
- Enter a ‘Label’ for your token that you can easily remember and click ‘Create.’
- Click ‘Copy to clipboard,’ then paste the token to the “password” field in the JIRA integration dialog.
Now that we have connected Ranorex Webtestit to Jira, we can add an endpoint to run our test. An endpoint is a set of configurations about the runtime environment of your test, such as which browser or operating system to use, and much more. In Ranorex Webtestit you can define local, remote, or custom endpoints to run your tests.
Save the endpoint and run the test. Upon completion of the test execution, a test report will be generated. If the test case has failed, we can create a Jira issue with a single click.
To do so, we’ll open the report and navigate to the failed test case, expand it and click the ‘Create Issue’ button.
This allows us to open the Jira issue in our default browser.
Ranorex Webtestit also allows us to resolve the issue directly from the test report, for example, when the same test case was successful after a repeated test run. In this case, simply click the ‘Resolve Issue’ button to close the Jira issue.
Of course, it’s not uncommon for bugs to show up again. For this reason, we can reopen the issue by clicking the ‘Reopen Issue’ button in the test report. However, your Jira project must support this workflow transition. Again, make sure that both ‘Active’ and ‘Completed’ status in the Preference dialog match the values set in your Jira project to allow reopening issues.
It should be noted that test cases in Ranorex Webtestit can only connect to one Jira issue at a time. The link information (additional data) is stored next to the test case in the form of a code comment, which includes all necessary information and can be edited if necessary. The link information will remain in the test case code even after the issue has been resolved.
Conclusion
In this example, we demonstrated how easily you can simplify and speed up your test workflow with the ability to generate JIRA issue tickets right from your test reports. This approach also enables you to resolve the issue from the test report as well as reopen the issue if the bug appears again.
Having test reports generated in a visually comprehensible format, as well as the integration with a JIRA project to create issue tickets right from those reports are just a few of the many benefits you get if you choose Ranorex Webtestit as your test automation tool.
You can download and install a FREE, full-featured trial of Ranorex Webtestit and learn more about the advantages of effortless testing.
