Azure Test Plans: Test Execution
This is my third article on Azure DevOps Test Plans best practices. In the first two I talk about Test cases and Test plans, so now we know how to write a test case and add it to a test plan so lets talk about how to execute those test plans and report on their results.
There are basically 2 ways to execute a test case, or in others words report if the test case is passing or failing.
- You can just simply select the test case and mark it as passed or failed.
- Or you can use the test runner to execute a group of test cases as it exposes the steps for you to execute.
The first option is quick and easy. Simply open your Test plan, select the test cases you want to report on and change their state from the drop down list at the top.
The test runner is useful when someone is writing the test cases and test plans, and there is an army of testers that will execute them. From your test plan, again select the test cases you want to execute and use the other drop down to start the test runner. There are several options to run a web version of the test runner vs a client version. Run with options lets you select the build you are testing. For the purpose of this article lets just execute the web version of the test runner. The result is the same.
When the test runner is started the test cases you selected to execute will be presented to you.
From the test runner you can mark each step as passed or failed. Officially the only steps that needs to be marked as pass or fail are those with an expected result. You can add comments along the way and take screen shots. You can even record a video of you executing the tests.
You can also add a bug right from the test runner.
Notice when you add a bug the Repro Steps are filled in for you from the test case steps. The build you were testing is added to the Found in Build field, and the test case and story you were testing are associated to the bug. In case you are wondering, the story comes from the test suite in the test plan.
Test Runs
Each time you execute one or more test cases, a test run is created. Test runs are the history of your test execution. You can see your test runs from the Runs tab under Test Plans.
By default this view will show you recent test runs. (runs created in the last 7 days). Click the filter tab above the list of test runs to see this query and change it. For example if you want to see the test runs for a particular build add a clause to the query to filter by build number.
When you open a test run you can add attachments and comments to the run.
Switch to the Test results tab to see the individual test cases that were executed in this run. You can add meta data to each test case including comments, attachments, failure type and resolution. Notice all the other information attached to the test case result, including run by, build, test plan, suite and test case and configuration.
You might be thinking, this is great but what is the current state of all the test cases in my test plan. In the last article on test plans I talked about charts that can be created to show the outcome of a test plan.
However there is another report that you can use that will allow you to select the results of multiple test plans at once. Showing you the current state of all the test cases in each selected test plan.
Navigate your way over to the Progress Report under Test Plans and take a look.
