Managing failed tests. Retrieve console logs and screenshots.
In UI tests, gathering detailed information about failed tests is crucial. Since tests often run independently using CI/CD and failures are investigated later, methods like console logs and screenshots are invaluable.
Console logs provide a detailed record of events, including JavaScript errors and network requests, which help in diagnosing issues. They allow developers to trace what went wrong during the test execution.
Screenshots offer a visual representation of the application’s state at the moment of failure. They can reveal UI issues such as layout problems or missing elements that logs might not show. By examining these screenshots, developers can quickly identify and resolve visual discrepancies.
Integrating these techniques into your testing framework streamlines the debugging process, leading to quicker identification and resolution of issues, resulting in more reliable software.
Below is an example of a simple implementation for handling such events.
Create a simple project and add dependencies.
The list of dependencies is as follows:
dependencies {
testImplementation platform('org.junit:junit-bom:5.9.1')
testImplementation 'org.junit.jupiter:junit-jupiter'
implementation 'org.seleniumhq.selenium:selenium-java:4.9.1'
implementation group: 'io.github.bonigarcia', name: 'webdrivermanager', version: '5.6.2'
testImplementation "org.projectlombok:lombok:1.18.22"
testAnnotationProcessor 'org.projectlombok:lombok:1.18.22'
implementation "org.slf4j:slf4j-simple:2.0.6"
}Add a class that will create a WebDriver (in my case, I use a dependency to avoid downloading the ChromeDriver manually).
package webdriver;
import io.github.bonigarcia.wdm.WebDriverManager;
import lombok.extern.slf4j.Slf4j;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.chrome.ChromeOptions;
import java.time.Duration;
@Slf4j
public class CreateWebDriver {
public WebDriver createLocalDriver() {
log.info("Try create webdriver");
WebDriverManager.chromedriver().setup();
ChromeOptions options = new ChromeOptions();
options.addArguments("--remote-allow-origins=*");
WebDriver driver = new ChromeDriver(options);
driver.manage().window().maximize();
driver.manage().timeouts().implicitlyWait(Duration.ofSeconds(10));
log.info("Webdriver created");
return driver;
}
}The ScreenshotWatcher class looks like this.
Add a class that will inherit from TestWatcher and actually catch our failed tests.
package utils;
import lombok.extern.slf4j.Slf4j;
import org.junit.jupiter.api.extension.ExtensionContext;
import org.junit.jupiter.api.extension.TestWatcher;
import org.openqa.selenium.OutputType;
import org.openqa.selenium.TakesScreenshot;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebDriverException;
import java.io.File;
import java.io.FileOutputStream;
import java.io.IOException;
import java.util.Optional;
@Slf4j
public class ScreenshotWatcher implements TestWatcher {
String path;
WebDriver driver;
ConsoleLogErrors consoleLogErrors;
public ScreenshotWatcher(WebDriver driver) {
this.path = "./build/test-results/test";
this.driver = driver;
consoleLogErrors = new ConsoleLogErrors(driver);
}
@Override
public void testAborted(ExtensionContext context, Throwable throwable) {
consoleLogErrors.getErrorsFromConsole();
consoleLogErrors.clearConsoleErrors();
}
@Override
public void testDisabled(ExtensionContext context, Optional<String> optional) {
optional.ifPresent(r -> log.info("Reason: {}", r));
}
@Override
public void testFailed(ExtensionContext context, Throwable throwable) {
consoleLogErrors.getErrorsFromConsole();
consoleLogErrors.clearConsoleErrors();
captureScreenshot(driver, context);
}
@Override
public void testSuccessful(ExtensionContext extensionContext) {
consoleLogErrors.clearConsoleErrors();
}
public void captureScreenshot(WebDriver driver, ExtensionContext context) {
String pathToFile = path + File.separator + context.getRequiredTestClass().getName();
String fullPath = pathToFile + File.separator + context.getDisplayName() + ".png";
try {
new File(pathToFile).mkdirs();
try (FileOutputStream out = new FileOutputStream(fullPath)) {
out.write(((TakesScreenshot) driver).getScreenshotAs(OutputType.BYTES));
}
} catch (IOException | WebDriverException e) {
log.info("screenshot failed:" + e.getMessage());
}
}
}The ConsoleLogErrors class.
This is a class that will read logs from the console and output them when necessary.
package utils;
import lombok.extern.slf4j.Slf4j;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.logging.LogEntry;
import org.openqa.selenium.logging.LogType;
import java.util.ArrayList;
import java.util.List;
import java.util.Set;
import java.util.function.Predicate;
@Slf4j
public class ConsoleLogErrors {
WebDriver driver;
public static List<String> errors = new ArrayList<>();
public ConsoleLogErrors(WebDriver driver) {
this.driver = driver;
}
public void getErrorsFromConsole() {
Set<String> logTypes = driver.manage().logs().getAvailableLogTypes();
List<LogEntry> toPrint;
for (String logType : logTypes) {
List<LogEntry> allLog = driver.manage().logs().get(logType).getAll();
if (allLog.isEmpty()) {
continue;
}
toPrint = allLog.stream().
filter(Predicate.not(x -> x.getMessage().contains("500"))).
filter(Predicate.not(x -> x.getMessage().contains("Any part of the error to skip"))).toList();
toPrint.stream().map(LogEntry::getMessage).forEach(errors::add);
if (!errors.isEmpty()) {
log.info("===============CONSOLE LOG=================");
log.info("{} ERRORS:", logType.toUpperCase());
errors.forEach(log::info);
log.info("-----------------------------------------");
}
}
}
public void clearConsoleErrors() {
driver.manage().logs().get(LogType.BROWSER).getAll();
errors.clear();
}
}The Test class.
The last thing, of course, is to add a class with tests, with which we can verify how everything works.
package tests;
import lombok.extern.slf4j.Slf4j;
import org.junit.jupiter.api.*;
import org.junit.jupiter.api.extension.RegisterExtension;
import org.openqa.selenium.WebDriver;
import utils.ScreenshotWatcher;
import webdriver.CreateWebDriver;
@Slf4j
@TestMethodOrder(MethodOrderer.OrderAnnotation.class)
public class ExampleTests {
public static WebDriver driver;
@RegisterExtension
public ScreenshotWatcher watcher = new ScreenshotWatcher(driver);
@BeforeAll
static void setUp() {
driver = new CreateWebDriver().createLocalDriver();
}
@Test
@Order(1)
@DisplayName("Passed test")
void examplePassed() {
log.info("Success test start");
driver.navigate().to("https://amazon.com/");
log.info("Success test finish");
}
@Test
@Order(2)
@DisplayName("Failed test")
void exampleFailed() {
log.info("Failed test start");
driver.navigate().to("https://google.com/error");
Assertions.fail("Caught error");
}
@Test
@Order(3)
@DisplayName("Handling console error 500")
void exampleHandling() {
log.info("500 error test start");
driver.navigate().to("https://office.com/error");
Assertions.fail("Caught error 500");
}
@AfterAll
static void tearDown() {
if (driver != null) {
log.info("Webdriver exists, I'll try to close it");
driver.quit();
}
}
}As a result, we obtain screenshots of failed tests located in the “build” folder.
And we also see the output from the browser console in our logs.
Additionally, in the ConsoleLogErrors class, there is the ability to skip certain logs if they are not of interest to us.
toPrint = allLog.stream().
filter(Predicate.not(x -> x.getMessage().contains("500"))).
filter(Predicate.not(x -> x.getMessage().contains("Any part of the error to skip"))).toList();In our example, we skip the error that contains the text “500”. As seen from the screenshot above, we only output logs for the test where there is a “404” error in the console.
Conclusion
Automated testing is crucial for software reliability. However, diagnosing failed tests can be challenging. This article shows how to improve test case investigations by using browser console logs and screenshots. Capturing detailed logs helps trace errors, while screenshots provide visual context for UI issues. Implementing these techniques speeds up troubleshooting and boosts productivity.
For more information and practical examples, visit my GitHub project.
