From a newbie to a test automation rock star, vol. 5: Debugging the automated test or how to fight the boredom of perfection

Tihomir Babić
Nov 26, 2019 · 6 min read
Image by Chris Reading from Pixabay

As a regular reader of this articles’ series, you know that we’ve already created two automated tests. Or should I say one, and then we built on it to make it more interesting. First, we created a simple test using the TestProject recorder. Then we enhanced this test by additional steps, but we’ve done that in a more complicated and time-consuming way. You’ve probably noticed that all our tests passed without problems. That’s a little boring, isn’t it? It’s not the point to find no problems. The point is to find them and solve them. As Rene Descartes famously put it:

„ Each problem that I solved became a rule, which served afterwards to solve other problems.“

But what should we do if something goes wrong while testing? That’s what we’ll learn in this article: how to use a reporting dashboard and debug our failed test.

Adding steps to an existing test

Let’s login to our TestProject site. Again, you will see our TestProject rockstar project.

Image by Tihomir Babić/screenshot

Click on it to get to our one and only FirstWeb Test.

Image by Tihomir Babić/screenshot

Click on it so we can see the steps comprising our test.

Image by Tihomir Babić/screenshot

Now we’re going to add one step to our test that will intentionally fail the test. First, we’re going to check the settings of our test. To do that, click on the „More” button in the upper right corner.

Image by Tihomir Babić/screenshot

Selecting the „Settings” will show you these options.

Image by Tihomir Babić/screenshot

„Default On Failure Behaviour“ should be set to „Fail Test“. If it’s not, please change that. The default test timeout is set to 15000ms (15 seconds), which is quite long. Let’s change it to 1000ms.

Image by Tihomir Babić/screenshot

Save the changes and they will be applied to the whole test. Let’s now create the step that will fail our test! Click on + sign to add a step. Adding the steps that way was something we’ve practiced in our latest article. Let’s select the element.

Image by Tihomir Babić/screenshot

In our search find the name textbox, it will be there since we used it last time. In case you just tuned in, please refer to the previous article for instructions on how to create an element.

Image by Tihomir Babić/screenshot

After selecting the right element, now select the action.

Image by Tihomir Babić/screenshot

Now try to find „Contains text?“ action and select it.

Image by Tihomir Babić/screenshot

As input parameters, we will type in „John Brown”. The action will check if in the name textbox there is „John Brown” typed in. Since in the previous step we have instructed the test to type in „James Brown”, this will, hopefully, fail the test. The created step is now last on the list.

Image by Tihomir Babić/screenshot

Let’s move this step up the pecking order. Simply drag it and drop it under step 3.

Image by Tihomir Babić/screenshot

Save & Exit the test now.

Image by Tihomir Babić/screenshot

Start the test, click the „Run“ button.

Image by Tihomir Babić/screenshot

Select the desired browser. In my case, it’ll be Chrome.

Image by Tihomir Babić/screenshot

Click „Next“, then the option „Override default input parameters“ should be selected, and then „Run“.

Image by Tihomir Babić/screenshot

After the test has finished let’s hover over the „Reports” option and see how did it go.

Image by Tihomir Babić/screenshot

Yes, it was a success! The test failed! Let’s click on it to see the report.

Image by Tihomir Babić/screenshot

In the central part, you can see the details of the report, which you can also download in PDF format. If you do that and open, you will see the report looks nice. It has all the important data that could help you to debug the test. There’s also a screenshot of the test where the test failed, which is helpful. That way we can see there’s is James Brown in the name textbox and we checked if the name is John Brown, which is wrong.

Image by Tihomir Babić/screenshot

If we go back to the report screen, there’s a pie chart that says the test passed 17%. If you hover over the different colors of the pie chart, you can see the percentages for different statuses:

· passed

· failed

· skipped

We can also very easily determine where the test failed using the report screen. You can see on the right side of the screen that 3 steps were executed and are marked green. Step 4 is red, which means it failed. All the subsequent steps were not executed. You could also click the „Show only failed“ toggle to show only failed steps of your test.

If you click on the failed test, you can see some details.

Image by Tihomir Babić/screenshot

Let’s now go back to the test, click on it and start the recorder.

Image by Tihomir Babić/screenshot

Once the recorder opens, let’s go to step 4, select „More”, and then „Run until here”. This means the test will be performed until step 4.

Image by Tihomir Babić/screenshot

We can see that step 4 is again red.

Image by Tihomir Babić/screenshot

We’ve already explained why this test fails. Now we’re going to correct this step and see if it helps the test to pass. Let’s click on the step.

Image by Tihomir Babić/screenshot

Now we’re going to change the input parameters from „John Brown” to „James Brown”.

Image by Tihomir Babić/screenshot

Click „Save”. Let’s now again select „More” on step 4 and select the option „Run from here”. This will start the test from step 4.

Image by Tihomir Babić/screenshot

When you run it, the test should perform until the latest step and all of them should pass.

Image by Tihomir Babić/screenshot

You can run the test from the TestProject project area. If you hover over Reports after that, you’ll see the test is green, which means it passed! Yes, we’ve successfully debugged our test!

Image by Tihomir Babić/screenshot

Conclusion

In this article, we’ve learned how to use reports in TestProject. It’s not too exciting when everything is OK, so we intentionally created an additional step to fail the test. That way we’ve learned how to use reports to locate the error and correct it. Now it feels a little more real because the whole point of test automation is to find mistakes! Perfect is boring, right?

Let’s keep on rockin’ in an automated world! And looking for other peoples’ mistakes.

Tihomir Babić

Written by

Data Analyst and Reporting Expert turned Database Designer and Developer from Zagreb, Croatia. Recently started learning about test automation.

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade