Task and Test Management Tool — The Way We Boost Our Jira
Hi! My name is Valery Lopakov and I am a QA Engineer at Rosberry. In today’s article I want to give a very quick overview of how we came to use a well-known task manager in a custom way.
At least 7 years out of 11 we have been working over full-scale mobile app development projects with their average duration being 5 to 6 months. Of course these projects have involved all types of testing on the way. Moreover lots of them live a much longer life — we support, update and enhance our apps for at least 3 to 5 years after release. Accordingly, one of the ongoing tasks of the QA Department at Rosberry has always been ensuring that testing and its results are properly documented. This job not only facilitates acceptance of works, project handover to a client’s product team (if there is any), but also makes all current and future activities of the QA Department and the company in general more structured and knowledgeable.
Needless to say that over the years we have tried various approaches to bug tracking and documenting testing artefacts, as well as various off the shelf software products and methodologies:
- Acceptance testing.
- Checklists in TestLink, Google Sheets, etc.
- Test cases in Sitechco, TM4J, Zephyr and TestRails.
- Etc.
Unfortunately, all of the above did not suit us for a number of specific reasons. For example, Test Rail, TM4J, Zephyr do not have any options to automate certain activities we usually carry out: bug ticket autofilling at its initial creation, changing the task status when it’s ready for testing, autoscoping for different types of testing, etc. Also, those software products are not fully intended to ensure that QA guys and developers are in one and the same working and information environment which is absolutely important to us.
So, choosing a solution that would help us overcome all the limitations, we were guided by several mandatory parameters that an applicable new Testing Management System would have to meet:
- One working and information space with developers (documentation and tasks).
- Conducive to automation.
- Flexible customization options.
- Convenient reporting.
- Integration with our time tracker.
You may be surprised but after all those years of ‘involuntary R&D’ we chose to go with the universally known Jira. But, as you understand, in its original form, Jira is hardly suitable for the role of a Test Management System. To have a viable product for conducting in-process and regression tests, maintaining project documentation and reporting, we had to invest certain efforts to customize it, and, you know what? We succeeded! We managed to automate our core daily processes, and that allowed us to devote more time to the testing process itself or writing test scenarios, rather than being distracted by non-core activities. But first things first — let’s see what we have now in place? At Rosberry we have been using Scrum for quite a long time, so the lingo is framework-specific .
Sprint & Boards
At the start of any project, we create three separate boards in Jira — Development, Functional Testing and Regression Testing. First off, this approach allows us to be on the same page with our developers, as the boards are in one and the same environment and are interdependent. Second off, having these three boards in place we can properly distribute testing tickets throughout development process and set the tickets subject to Regression Testing (which is usually at the final stage of the project) apart. The Regression Testing board now comprises only those tickets which have gone through all the preliminary in-process testing.
We use two-week sprints. On Monday the team finalizes the work scope on their board and starts development. Doing the job, developers move tasks to the Test Build on their board, and a dedicated QA Engineer tests the tasks having them automatically moved to their own board, reports bugs or sends the tasks to Done. Some automations we’ve introduced so far to simplify the work at this stage are the following:
- Reporting a bug. When a bug ticket is created, there pops up a table with certain columns and rows: Device, Build, Steps to Reproduce, Expected and Actual Results.
- Creating a subtask. When a subtask is created within a task, there appears the same component as with the parent task.
- Dragging and dropping into the Test Build. When a User Story is moved to the Test Build (ready for testing), a scenario ticket is automatically added to the current testing sprint, and if a user story or a development task does not have a scenario ticket associated with it, it is simply copied to the testing sprint.
- Dragging a task to Done. When a scenario ticket is moved to Tested, the development task or a user story associated with it is automatically moved to Done, but only if there were no failed scenarios or sub-bugs for the task.
Scenarios
Since I’ve used the term ‘scenario’ several times, I want to shortly explain how we understand it and what we are using scenarios for.
About two years ago we made the BDD (behavior-driven development) methodology an integral part of our development process. If you want to learn more about it, you can read this article. Yet in outline a BDD-scenario is a written description of your product’s behavior from one or more users’ perspectives. BDD scenarios tend to follow a specific format with 3 obligatory words used: GIVEN, WHEN, THEN. These 3 elements help to describe the behaviour of the system using context, actions and outcomes. Scenarios are designed to reduce the cost of ‘translation’ and make it easier for the developers to understand the requirements and for the QA to test everything properly.
BDD scenarios are grouped around one user story, which refers to one or another part of the functionality.
A user story is a conditional name of a part of the application functionality with a description of the value for the user, administrator or owner of the application.
All stories within the same functionality group are combined into epics.
An epic is a part of the functionality of a mobile application that combines several user stories.
Thus, the hierarchy of tickets in our Jira now is as follows: Project — Epic — User Story — BDD Scenario.
For example, DS — Home Page — First Launch — The user launches the application for the first time
As you understand, an average project includes multiple user stories and a whole bunch of BDD scenarios which in our case turn out to be the principal project documentation. Since Jira is a part of the Atlassian product family, we keep and maintain our documentation in Confluence. Everything is stored as Task Tickets containing a link to certain user stories and scenarios in Jira. What is convenient is that any ongoing change in a Task Ticket will automatically be there both in Jira and Confluence. This makes it possible to use the same scenarios as both project and testing documentation.
And now back again to our customized Jira. At the end of the second week of the sprint, there comes a moment we call “The Code Freeze”. After that no changes could be made to the code of the current increment, and the so-called Testing Run starts. The Testing Run in our lingo is the kind of ‘mini-regression test’ that is carried out only for the features of the current and previous sprints checked against major in-sprint scenarios only. Actually, we’ve made our Jira to automatically offer the scope for the Testing Run, and a QA Engineer does not waste time anymore compiling a separate board — s/he just tests.
After the Testing Run is over Jira automatically moves to the backlog those tasks which have been checked twice, but leaves those that have been checked once for the next Testing Run. To make all of the above possible, we’ve introduced the following automations:
- Completion of Testing Sprint. When the testing sprint is over, the system checks if there are any failed scenarios. If it finds such scenarios, they are moved to the next testing sprint to rerun. Successful scenarios are automatically grouped together to be checked during the Testing Run.
- Completion of Testing Run. When the Testing Run ends, the system checks which tasks were tested twice and moves them into the Regression Backlog (a part of QA Regression Board I mentioned before). Those tested once remain in the sprint to be tested again during the next Testing Run.
On Friday, at the end of a two-week sprint, an in-sprint test report is compiled and a sprint retrospective is held. To help us understand the sprint progress and to successfully hold a retrospective, we’ve come up with some additional Jira automations:
- Generating an automated report. Jira now has all necessary data to prepare a detailed report on the work of the QA Department and testing results during the sprint. This report even includes the amount of time dedicated to working on a particular task, this becomes possible due to integration of our time tracker into Jira.
When we finally go over to the Regression Testing, we have at least two rounds (primary and final), for each of these rounds Jira generates a report. This report is based on a standard QA Work Report during the sprint, so most of the data for it is obtained from an automated Jira report.
Bottomline
Of course, a quick overview can hardly give a comprehensive idea of how it all works now with us. Yet, what we can definitely say is that Jira over time can really turn into a powerful and flexible Test Management System, capable of not only storing test cases in the form of BDD scenarios, but can also be flexibly integrated with the task management board for development, the Сonfluence documentation repository and a time tracker. We now know that Jira is more than conducive to different automations which can definitely facilitate routine activities sparing more time for application research and testing.
