Test Management with TestRail

Checkout.com is a fast growing company. Our teams are becoming bigger, and the number of internal and external projects increases too. Many of the projects within the company are integrated with each other and require good collaboration between team members, especially QA engineers.

Testing is an important part of our development process. Test documentation and testing results should be accessible to many people: QA engineers, developers, project managers, product analytics and others. When our testing community has grown significantly, we have faced the problem of how to make our testing processes more transparent for all the participants.

We used a number of tools before, that helped us to solve various problems. Like test documentation in Confluence, test results in Cypress dashboard for UI testing, and some other distributed solutions. But then it has become too elaborate to maintain all of these sources up to date and aligned with each other. At this point, we switched to Testrail as a main tool.

Testrail is a test management tool that is used by a lot of companies. It’s a web-based application that gives its users very wide functionality to organise test processes. But most importantly it has an external API that allows for connecting test automation to it and is compatible with a variety of languages.

We started using Testrail at the beginning of 2020 with only one or two projects as an experiment. And by the end of the year, it has become our main test management tool at Checkout.com. Here I am going to list the key advantages that Testrail brought to our testing:

Well organised, flexible and readable test documentation

At Checkout.com, projects can be very different and so are the testing approaches. Therefore documentation can be organised in many ways. Some teams have it as one big piece that contains all the scenarios. Some teams may need to split documentation into smaller parts under which scenarios are combined together. All of these options are possible with Testrail.

Integration with test automation

Many of our UI and API tests are written in NodeJS, so we use a direct API to connect them to Testrail. This API can be used in a variety of programming languages. It aligns code strictly with the respective tests written in test suites. Even though not all the scenarios may be covered by test automation, test results may be added manually via Testrail UI interface.

Easy planning and reporting within the test community

Planning is a great, but not very obvious, part of Testrail functionality. It may not be widely used across teams, but it is still important. It allows to distribute testing efforts between team members and also gives an opportunity to create milestones under which test results can be organised together.

All the results can be shared in multiple ways as a variety of reports is provided. It can be milestones, test runs, test suites, user activity reports and many others. If this part is well organised it can bring great value as it allows users to navigate across implemented changes in a certain project.

Integration with other tools

There are many tools that Testrail can be integrated with, like Jira or Confluence. That allows you to use Testrail in your own custom set of tools that are required for your testing purposes.

Customisation for the company’s needs

Testrail users are not limited by default features. Some functionality as templates, priorities, statuses and others can be customised to better fit a company’s needs.

Also, access to certain parts of testing can be restricted for some users if additional security measures are required. For example, it is possible to allow access to project info only for your team members and no one else.

It’s worth saying that Testrail is not our only reporting instrument. We still use other tools that provide test results, but today they are mostly used in addition to Testrail. Some of them are integrated into our deployment process and are visible to all team members (like tests in Teamcity), others are only being run locally and available for testers only. But visibility and connection between documentation and test results make Testrail our main instrument. We constantly customise Testrail so it fits our needs. It allowed us to unify requirements for test documentation and keep it in one place for the whole company. This also allows us to spend less time on documentation maintenance and focus more on testing itself. And we encourage our teams to use Testrail more at all stages of the testing process to make it more visible and transparent.