Capture screenshot on test failure for parallel execution with Appium and TestNG

This is a tutorial post.
If you need just the solution you can jump at this topic, but I strongly recommend you to read the introduction.

Introduction

One of basic test automation architecture item is know when your test fails and have a kind of evidence of it. Commonly we use, for front-end tests, a screenshot to show where the error ocurred.

Mainly the screenshot name is the test name. Some people add date and time to track this information, but for parallel execution we need to add more information on the screenshot name.

The example will show an Appium test script developed with Java and the usage of TestNG to facilitate the approach to capture a screenshot.

The basics

How to capture screenshot with Appium

In my case, that I'll save a file in a directory, I need to call the getScreenShotAs method from driver, an pass the parameter OutputType.FILE.
After use the FileUtils class from Commons IO to copy our file (screenshot) to a physical directory.

File file = driver.getScreenshotAs(OutputType.FILE);
FileUtils.copyFile(file, new File("myScreenshot.png");

How to add it to your test

Basically you'll add the lines above to some strategy on test failure.
The basic one it add it to a try-catch block. It's "ugly", but it works.

// some code ommited
try {
// add the interactions
} catch (Throwable t) {
// capture the screenshot
// fail the testcase
}

The problem with this approach is that you need to add this block in every testcase (automated script) and, depends how many tests you have, it is a lot of work. And remember DRY.

Problem about the parallel test execution

If you running your test in parallel a solution about how you will differentiate the screenshot file name.

Using the approach to give the test class name or test name will cause a file override. Using the approach adding the date and time will cause a manual work to see which device that screenshot belongs.

The solution

Create a Test Listener

In TestNG you can implement some listeners to modify TestNG's behavior.
The ITestListener (doc | javadoc) add a capability to be notified, in realtime, about test starts, passes, fail, etc…

This interface (ITestListener) have a method onTestFailure that will be called every time your test fail. So we have the listener to add the screenshot generation.

Simply create a class and implements ITestListener

public class TestListener implements ITestListener {

@Override
public void onTestStart(ITestResult iTestResult) {}

@Override
public void onTestSuccess(ITestResult iTestResult) {}

@Override
public void onTestFailure(ITestResult iTestResult) {}

@Override
public void onTestSkipped(ITestResult iTestResult) {}

@Override
public void onTestFailedButWithinSuccessPercentage(ITestResult iTestResult) {}

@Override
public void onStart(ITestContext iTestContext) {}

@Override
public void onFinish(ITestContext iTestContext) {}

}

Add the screenshot capture onTestFailure

It's easy to add a screenshot on test failure. Just add the screenshot code (you have seen this above) on the onTestFailure method!

But you'll face the same problem I had: how to capture the driver object to use in the getScreenshotAs method.
The simple snnipet, until now is:

@Override
public void onTestFailure(ITestResult iTestResult) {
File file = driver.getScreenshotAs(OutputType.FILE);
FileUtils.copyFile(file, new File("myScreenshot.png");
}

How to get the driver object in the TestListener class

With the iTestResult information (coming from the parameter) you can access the test class where the error occured. Basically it's a reflection.

Basically you need to get the class, get the field (the driver) and the value of this field (driver instantiated). The driver is on the test, and not on a base class or something related, because of the paralellism.

The basic snnipet for this is:

@Override
public void onTestFailure(ITestResult iTestResult) {
Class clazz = iTestResult.getTestClass().getRealClass();
Field field = clazz.getDeclaredField("driver");

field.setAccessible(true);

AppiumDriver<?> driver = (AppiumDriver<?>) field.get(iTestResult.getInstance());
}

Now you just need to add the code for capture the screenshot. We're almost there!

How to associate the listener with the test

Simply add an annotation @Listeners passing as parameter you custom listener class.

@Listeners(TestListener.class)
public class MyTest {
}

Now every time your test fails, and screenshot will be taken.

How to solve the screenshot name problem

Now, for the complete example, we need to solve this problem.
As we use parameters on the test to execute the same test on different devices a trick is add these parameters on the screenshot file name.

It's also a good practice (or recommended practice) to add the test class and test name on the screenshot file name.

So I've created a helper method to generate the filename:

private String composeTestName(ITestResult iTestResult) {
StringBuffer completeFileName = new StringBuffer();
    completeFileName.append(iTestResult.getTestClass().getRealClass().getSimpleName()); // simplified class name
completeFileName.append("_");
completeFileName.append(iTestResult.getName()); // method name

// all the parameters information
Object[] parameters = iTestResult.getParameters();
for(Object parameter : parameters) {
completeFileName.append("_");
completeFileName.append(parameter);
}

// return the complete name and replace : by - (in the case the emulator have port as device name)
return completeFileName.toString().replace(":", "-");
}

An example of a screenshot file name with this code is: MyTest_myTest_android_emulator-5554_7.0.1.png

A complete example

You can see a complete example, with the basic architecture to execute parallel tests with Appium and this trick to create screenshots on test failure.

The complete test listener can be fount at https://github.com/eliasnogueira/appium-parallel-execution/blob/master/src/main/java/com/eliasnogueira/support/TestListener.java

Any questions? Fill a comment :)