Rerun failed testcases in selenium using TestNG

  1. By using testng-failed.xml file in test-output folder
  2. By implementing TestNG IRetryAnalyzer
  • Create a java class with tests
package testngDemo;
import org.testng.Assert;
import org.testng.annotations.Test;
public class RerunFailedTest1 {
@Test
public void test1() {
Assert.assertTrue(true);
}
@Test
public void failedtest1() {
Assert.assertTrue(false);
}
}
  • Create a testng.xml file: Right click on project=> TestNG=> Convert to TestNG
  • Now run the testng.xml file: Right Click=> Run As=>TestNG Suite
  • Now the console output is as below with 3 failed tests.
  • Create an implemented class of IRetryAnalyzer interface and override retry method:
package testngDemo;
import org.testng.IRetryAnalyzer;
import org.testng.ITestResult;
public class RetryAnalyzer implements IRetryAnalyzer {
//Counter to keep track of retry attempts
int retryAttemptsCounter = 0;
//The max limit to retry running of failed test cases
//Set the value to the number of times we want to retry
int maxRetryLimit = 3;
//Method to attempt retries for failure tests
public boolean retry(ITestResult result) {
if (!result.isSuccess()) {
if(retryAttemptsCounter < maxRetryLimit){
retryAttemptsCounter++;
return true;
}
}
return false;
}
}
  • Now add an attribute called “retryAnalyzer” to @Test annotated method with value as Implemented class name as shown below
package testngDemo;
import org.testng.Assert;
import org.testng.annotations.Test;
public class RerunFailedTest1 {
int counter;

@Test
public void test1() {
System.out.println("test run");
Assert.assertTrue(true);
}
@Test(retryAnalyzer = RetryAnalyzer.class)
public void failedtest1() {

System.out.println("Failed test run");
Assert.assertTrue(false);
}
}
  • Now if we run TestNG.xml we see failed test case is executed one more time as we gave retry count= 1. Test case is marked as failed only after it reaches max retry count. In the remaining test runs it will be marked as skipped, the console output is shown below.
  • Let’s consider a scenario where failed test case passed after 2 reruns, the code is as shown below. so we are changing retry count = 3 in RetryAnalyzer class.
package testngDemo;
import org.testng.Assert;
import org.testng.annotations.Test;
public class RerunFailedTest1 {
int counter;

@Test
public void test1() {
System.out.println("test run");
Assert.assertTrue(true);

}

@Test(retryAnalyzer = RetryAnalyzer.class)
public void failedtest1() {
counter++;
if(counter <= 2)
{
System.out.println("Failed test run");
Assert.assertTrue(false);
}
System.out.println("Failed test run passed");
}
}
  • But here we need to write attribute to every testcase we want to rerun. This will be cumbersome when we have 100’s of testcases. To overcome this we can implement another TestNG interface IAnnotationTransformer that overrides transform method.
  • Create an implemented class of IAnnotationTransformer interface and override transform method as:
package testngDemo;
import java.lang.reflect.Constructor;
import java.lang.reflect.Method;
import org.testng.IAnnotationTransformer;
import org.testng.IRetryAnalyzer;
import org.testng.annotations.ITestAnnotation;
public class AnnotationTransfer implements IAnnotationTransformer{ //Overriding the transform method to set the RetryAnalyzer
public void transform(ITestAnnotation testAnnotation, Class testClass,
Constructor testConstructor, Method testMethod) {
IRetryAnalyzer retry = testAnnotation.getRetryAnalyzer();
if (retry == null)
testAnnotation.setRetryAnalyzer(RetryAnalyzer.class);
}
}
  • Add listeners to TestNG.xml file , so that we need not add attribute every test case.
  • The console output is same as previous one.

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store