Automated Visual Regression Testing with Playwright: Uncover Hidden Design Flaws

Visual regression testing (VRT) is a crucial part of web development that ensures the appearance and layout web application remains consistent over time. Manual VRT can be time-consuming and error-prone, which is why automating this process can greatly improve efficiency and accuracy. Playwright, a powerful open-source automation tool, provides a reliable solution for automating VRT. In this blog post, we will explore how to automate VRT using Playwright in Blibli.

VRT involves comparing screenshots of a web page or application taken at different points in time to identify any visual discrepancies. These discrepancies could be the result of unintended changes introduced during development, such as layout shifts, color variations, or broken images. Automating this process helps detect such issues early and ensure a consistent visual experience for users.

Playwright is a powerful automation library that allows developers to write tests for web applications across different browsers, including Chrome, Firefox, and WebKit. It provides a simple and intuitive API for automating user interactions, capturing screenshots, and performing visual comparisons.

Playwright architecture:

1. Playwright Core: At the heart of Playwright is the Playwright Core, which serves as the foundation for all browser automation capabilities. It provides a unified API for interacting with different web browsers: Chrome, Firefox, and WebKit. Playwright Core enables developers to write browser automation scripts in JavaScript or TypeScript. We currently using Playwright version of 1.17.1.
2. Browser Context: A Browser Context represents an isolated browsing session within a specific browser instance. It encapsulates cookies, cache, and other browser state information. Playwright allows the creation of multiple Browser Contexts, each with its own cookies and storage, which enables parallel testing and isolates test scenarios.
3. Browser: A Browser represents a browser instance, such as Chrome or Firefox. It manages the lifecycle of the browser process and provides methods for creating new Browser Contexts. Playwright supports running tests in multiple browsers simultaneously, allowing for cross-browser compatibility testing. For now we only use Chrome.
4. Page: A Page represents a single web page within a Browser Context. It provides methods for interacting with the page’s DOM, capturing screenshots, injecting JavaScript code, and simulating user input events. Playwright allows multiple pages to be created within a single Browser Context, enabling scenarios such as navigating between different pages or opening multiple tabs.
5. Playwright Drivers: Playwright leverages browser-specific drivers to communicate with the underlying browser instances. These drivers are responsible for translating Playwright’s unified API calls into browser-specific protocols. Playwright currently uses three separate drivers: Chromium, Firefox, and WebKit. Each driver is responsible for handling interactions with its respective browser.
6. Playwright CLI: Playwright provides a Command Line Interface (CLI) tool that allows developers to perform various tasks, such as installing or updating browser binaries, recording and generating code snippets, or running tests. The CLI tool simplifies the setup and execution of Playwright scripts and integrates well with popular testing frameworks.
7. Test Framework Integration: Playwright seamlessly integrates with popular test frameworks. We decide to use Jest for organizing and executing tests, generating reports, and handling assertions.

Using Playwright, you can navigate to specific web pages, interact with elements, and capture screenshots of the entire page or specific elements. By taking screenshots at different stages of development, you can establish a baseline for visual comparison.

It also comes with powerful tooling:

1. Code Generator, generate test by recording user action (flow) then save in javascript, ready to copy paste right away to test file
2. Code Inspector, inspect page, logs by accessing dev tools, see click points
3. Tracer Viewer, capture test progress (state, failures, timing) and post mortem analysis

Example of layout shifting detected. Left: image baseline. Center: differences detected and red highlighted. Right: image screenshot from latest test run.

But, we encounter discrepancies in rendering among different operating systems (Windows, Linux, MacOS) due to variations in the underlying system configurations and dependencies

Organizing and managing test cases is crucial for an effective VRT strategy. Playwright along with test frameworks — Jest that enable us to structure tests and generate meaningful reports.

To ensure visual regression tests are executed regularly, it’s essential to integrate them into your continuous integration and deployment (CI/CD) pipeline. By automating the execution of visual regression tests during each deployment, you can catch any visual regressions early and prevent them from reaching production.

Automating VRT with Playwright offers numerous benefits, including improved efficiency, accuracy, and early detection of visual discrepancies. By integrating Playwright into your testing workflow, you can ensure a consistent visual experience for your users and maintain the quality of your web applications. Start exploring the power of Playwright and take your VRT to the next level!