From a newbie to a test automation rock star, vol. 5: Debugging the automated test or how to fight the boredom of perfection
As a regular reader of this articles’ series, you know that we’ve already created two automated tests. Or should I say one, and then we built on it to make it more interesting. First, we created a simple test using the TestProject recorder. Then we enhanced this test by additional steps, but we’ve done that in a more complicated and time-consuming way. You’ve probably noticed that all our tests passed without problems. That’s a little boring, isn’t it? It’s not the point to find no problems. The point is to find them and solve them. As Rene Descartes famously put it:
„ Each problem that I solved became a rule, which served afterwards to solve other problems.“
But what should we do if something goes wrong while testing? That’s what we’ll learn in this article: how to use a reporting dashboard and debug our failed test.
Adding steps to an existing test
Let’s login to our TestProject site. Again, you will see our TestProject rockstar project.
Click on it to get to our one and only FirstWeb Test.
Click on it so we can see the steps comprising our test.
Now we’re going to add one step to our test that will intentionally fail the test. First, we’re going to check the settings of our test. To do that, click on the „More” button in the upper right corner.
Selecting the „Settings” will show you these options.
„Default On Failure Behaviour“ should be set to „Fail Test“. If it’s not, please change that. The default test timeout is set to 15000ms (15 seconds), which is quite long. Let’s change it to 1000ms.
Save the changes and they will be applied to the whole test. Let’s now create the step that will fail our test! Click on + sign to add a step. Adding the steps that way was something we’ve practiced in our latest article. Let’s select the element.
In our search find the name textbox, it will be there since we used it last time. In case you just tuned in, please refer to the previous article for instructions on how to create an element.
After selecting the right element, now select the action.
Now try to find „Contains text?“ action and select it.
As input parameters, we will type in „John Brown”. The action will check if in the name textbox there is „John Brown” typed in. Since in the previous step we have instructed the test to type in „James Brown”, this will, hopefully, fail the test. The created step is now last on the list.
Let’s move this step up the pecking order. Simply drag it and drop it under step 3.
Save & Exit the test now.
Start the test, click the „Run“ button.
Select the desired browser. In my case, it’ll be Chrome.
Click „Next“, then the option „Override default input parameters“ should be selected, and then „Run“.
After the test has finished let’s hover over the „Reports” option and see how did it go.
Yes, it was a success! The test failed! Let’s click on it to see the report.
In the central part, you can see the details of the report, which you can also download in PDF format. If you do that and open, you will see the report looks nice. It has all the important data that could help you to debug the test. There’s also a screenshot of the test where the test failed, which is helpful. That way we can see there’s is James Brown in the name textbox and we checked if the name is John Brown, which is wrong.
If we go back to the report screen, there’s a pie chart that says the test passed 17%. If you hover over the different colors of the pie chart, you can see the percentages for different statuses:
· passed
· failed
· skipped
We can also very easily determine where the test failed using the report screen. You can see on the right side of the screen that 3 steps were executed and are marked green. Step 4 is red, which means it failed. All the subsequent steps were not executed. You could also click the „Show only failed“ toggle to show only failed steps of your test.
If you click on the failed test, you can see some details.
Let’s now go back to the test, click on it and start the recorder.
Once the recorder opens, let’s go to step 4, select „More”, and then „Run until here”. This means the test will be performed until step 4.
We can see that step 4 is again red.
We’ve already explained why this test fails. Now we’re going to correct this step and see if it helps the test to pass. Let’s click on the step.
Now we’re going to change the input parameters from „John Brown” to „James Brown”.
Click „Save”. Let’s now again select „More” on step 4 and select the option „Run from here”. This will start the test from step 4.
When you run it, the test should perform until the latest step and all of them should pass.
You can run the test from the TestProject project area. If you hover over Reports after that, you’ll see the test is green, which means it passed! Yes, we’ve successfully debugged our test!
Conclusion
In this article, we’ve learned how to use reports in TestProject. It’s not too exciting when everything is OK, so we intentionally created an additional step to fail the test. That way we’ve learned how to use reports to locate the error and correct it. Now it feels a little more real because the whole point of test automation is to find mistakes! Perfect is boring, right?
Let’s keep on rockin’ in an automated world! And looking for other peoples’ mistakes.
