Achieving Cross-Browser & Responsive Coverage in Cypress Snapshot Testing
Snapshot testing is a powerful technique for verifying the visual appearance of a web application. It captures screenshots of components and pages and compares them against previously stored reference images to detect any unexpected changes that might affect user experience which is very important for us as Trendyol Tech Team. In this article, we will explore how we implemented snapshot testing with Cypress in Seller Base Team, focusing on two essential features: supporting multi-browser testing and ensuring responsiveness across different device screen sizes such as desktops, tablets, and phones. This development will help teams identify small UI differences and enhance their development cycle.
Why do we need Cross-Browser and responsive Coverage?
Let’s talk about why it’s really important to have features like snapshot testing with cross-browser and responsive coverage support for a web application. Here are some reasons:
- Diverse User Bases: In today’s digital age, users access websites through a multitude of browsers and devices, ranging from traditional desktops to tablets and phones. Each platform brings its unique nuances, affecting how a web application is rendered. Cross-browser & Responsive Coverage supported snapshot testing is essential to guarantee a consistent experience for all users, regardless of their preferred browsing tools.
- User Experience: A seamless user experience is a cornerstone of user satisfaction. Even minor inconsistencies or broken functionalities can lead to frustration and prompt users to avoid your application. By thoroughly testing across various browsers, you can identify and rectify issues.
- Effective Development Workflow: The ability to detect and address UI discrepancies early in the development cycle streamlines the overall process. Responsive coverage and cross-browser supported snapshot testing identify issues at their inception, reducing the time and effort needed for debugging in later stages.
How We Made It Happen?
Wondering how we achieved this? Here’s a breakdown of the steps you can follow to add Cross-Browser & Responsive Coverage support to your snapshot testing project.
Keep in mind, that this guide assumes you’ve already got your snapshot testing project up and running using Cypress. If you’re looking for detailed instructions on setting up your snapshot testing project, please visit Cypress’ official documentation.
1. Dynamic Custom Snapshots Directory:
When running snapshot tests on multiple browsers, slight variations in rendering can occur due to browser-specific behaviors. To address this, we’ll modify the commands.js file in our Cypress project to dynamically set a custom snapshots directory based on the used browser.
// commands.js
// Determine the current browser name
const browserName = Cypress.browser.name;
// Define custom snapshots directories for different browsers
const browserSnapshotDirs = {
electron: 'electron',
chrome: 'chrome',
firefox: 'firefox',
};
// Set the custom snapshots directory based on the current browser
const customSnapshotsDir = `cypress/snapshots/${browserSnapshotDirs[browserName]}`;
// Add the Match Image Snapshot command with the custom snapshots directory
addMatchImageSnapshotCommand({
customSnapshotsDir,
});With this configuration, Cypress will save snapshots for each browser under separate directories, allowing easy comparison and identification of any subtle visual differences across browsers.
2. Dockerizing Cypress Tests:
When we added our snapshot testing project to the UI project pipeline, we faced a problem. The tests were failing because the headless browsers had different screen sizes. This happened because Cypress makes the browsers act differently depending on the operating system.
The main issue was that our own local environment used macOS, but the pipeline jobs used Linux. To solve this, we turned to Docker. Docker helps us make a consistent testing environment that works the same way for both our own computers and the pipeline testing. This way, we make sure that our tests work well in all situations.
For seamless integration with continuous integration tools, we’ve devised a Dockerfile configuration that facilitates the execution of multiple commands within the ENTRYPOINT. This strategic approach empowers us to run Cypress tests for different browsers within a single Docker container, thereby ensuring uniformity across diverse testing scenarios.
# Dockerfile
# Use the Cypress official image as the base image
FROM cypress/included:10.4.0
# Copy package-lock.json, package.json, and .npmrc to the container
COPY package-lock.json package.json .npmrc ./
# Install project dependencies
RUN npm ci
# Verify Cypress installation
RUN npx cypress verify
# Copy the run-tests.sh script to the container
COPY run-tests.sh ./
# Set the entrypoint to run the run-tests.sh script
ENTRYPOINT ["/bin/bash", "./run-tests.sh"]3. Shell Script for Browser-Specific Execution:
Now we’ll create a new shell script called run-tests.sh, which will contain the necessary scripts to be executed in the Dockerfile for specific browsers. This script will be responsible for selecting the appropriate browser for testing.
# run-tests.sh
#!/bin/bash
# Check the BROWSER environment variable to determine the target browser
if [ "$BROWSER" = "firefox" ]; then
# If the target browser is Firefox, run Cypress tests for Firefox
npm run cy:run:firefox
else
# Otherwise, run Cypress tests for Chrome (assuming it's the default browser)
npm run cy:run:chrome
fiThis script allows choosing between running snapshot tests on Chrome and Firefox, according to the provided BROWSER environment variable.
4. Cypress Configuration for Viewport Sizes and Device Screens:
To ensure that tests run on different screen sizes, we’ll add a constants file named constants.js that defines supported device screens using Cypress ViewportPresets.
// constants.js
export const deviceSizes = [
'macbook-15',
'ipad-mini',
'iphone-xr'
];For more viewport preset options, please visit this link.
Next, we’ll create the Cypress configuration file (cypress.config.js) to include viewport sizes for different browsers such as Electron, Chrome, and Firefox.
// Define browser-specific viewport sizes
on('before:browser:launch', (browser = {}, launchOptions) => {
if (browser.name === 'electron' && browser.isHeadless) {
launchOptions.preferences['width'] = 1440;
launchOptions.preferences['height'] = 900;
launchOptions.preferences['resizable'] = false;
}
if (browser.name === 'chrome' && browser.isHeadless) {
launchOptions.args.push('--window-size=1440,900');
}
if (browser.name === 'firefox' && browser.isHeadless) {
launchOptions.args.push('--width=1440');
launchOptions.args.push('--height=900');
}
return launchOptions;
});5. Run Tests for All Devices:
To achieve complete screen size coverage, we’ll modify the test files to loop through all the device sizes and set the viewport accordingly before running each test. The following is an example of a snapshot test file.
// test.spec.js
import { deviceSizes } from './constants';
deviceSizes.forEach((deviceName) => {
describe(`Device Screen: ${deviceName}`, () => {
beforeEach(() => {
// Set the viewport to the current device size
cy.viewport(deviceName);
});
it('should open ComponentName when ButtonName button is clicked', () => {
cy.matchImageSnapshot(`PageName|FlowName|ComponentName|${deviceName}`);
});
});
});By running the snapshot tests with different viewport sizes, the development team can detect and address any responsiveness issues across various screen sizes, ensuring the best user experience.
6. Run Scripts for Different Browsers:
Now, we’ll add new run scripts to the package.json file, each targeting a specific browser. These scripts will be invoked by the run-tests.sh script with the appropriate browser parameter.
// package.json
"scripts": {
"cy:build:docker": "docker build . -f Dockerfile -t snapshot-testing",
"cy:run:chrome": "cypress run -b chrome",
"cy:run:firefox": "cypress run -b firefox"
}By having separate run scripts for Chrome and Firefox, the development team can effortlessly initiate snapshot testing on different browsers, simplifying their development workflow.
To run tests in the local environment for different headless browsers, you can add the following scripts under the previous scripts object.
"cy:run:docker:chrome": "docker run --rm -e BROWSER=chrome -v $PWD:/cypress -w /cypress snapshot-testing",
"cy:run:docker:firefox": "docker run --rm -e BROWSER=firefox -v $PWD:/cypress -w /cypress snapshot-testing",7. GitLab CI Configuration:
To ensure continuous testing and integration, we’ll create the .gitlab-ci.yml file to include jobs for running snapshot tests for each browser separately. These jobs will execute the run scripts defined in the previous step in parallel.
The following is an example of configuring a test job for Chrome browser in the GitLab pipeline. For the complete source code please visit the repository mentioned in the conclusion section below.
# .gitlab-ci.yml
# Job to run tests on Chrome browser
Test Chrome:
stage: Snapshot
image: docker-image-url
services:
- docker:VERSION
variables:
DOCKER_HOST: "tcp://localhost:PORT_NUMBER"
only:
- master
script:
- docker build . -f Dockerfile -t snapshot-testing
- docker run - rm -e BROWSER=chrome -v $PWD:/cypress -w /cypress snapshot-testing
artifacts:
# Specify that artifacts (snapshots) should be saved
when: always
paths:
- cypress/snapshots
expire_in: 15 daysWhat We Gained?
Our journey in implementing Cypress snapshot testing has been transformative for the Trendyol Seller Base Team. It’s not just about ensuring visual consistency; it’s about improving the overall development process and user experience.
- Expanded Testing Reach:
Our snapshot testing project has significantly expanded our testing reach. We now test our application on a wide array of browsers and screen sizes, ensuring that it looks and functions optimally on any device. - Early Issue Spotting:
One of the most significant benefits we’ve gained is the ability to spot UI issues early in the development process. In the past, minor discrepancies in how our application was rendered on different browsers and screen sizes often went unnoticed until they became critical. Now, with snapshot testing, we catch these issues at their inception, long before they can affect our users. - Seamless Pipeline:
Every update to the UI project initiates a round of snapshot tests. This acts as a valuable safety net, ensuring that only robust changes make their way to the live environment. Our development process is now more streamlined and efficient, thanks to this automated testing pipeline.
- Consistent Environment:
Docker has been a game-changer in maintaining a consistent testing environment. Previously, differences between local development environments and our pipeline sometimes led to false positives or false negatives in our tests. Docker has eliminated these discrepancies, ensuring that tests produce consistent and reliable results across all stages of development. - GitLab Harmony:
Our GitLab CI setup now handles tests smoothly and efficiently, running them for different browsers in parallel. This has significantly improved our development workflow. By automating the testing process and making it easy to test across various browsers, we’ve reduced manual testing efforts and accelerated our release cycle.
Conclusion:
By following the steps we’ve discussed in this article, your development team can easily set up Cypress snapshot testing. This will help you test your website on different web browsers, make sure it looks good on various screen sizes, and cover all the different sizes your site might be viewed on. This approach will help your team spot even the smallest differences in how things look while you’re working on your website, making sure it looks nice no matter how people are viewing it.
For the complete source code of the mentioned configuration files, please visit this repository.
Thanks for reading!
