Be Fearless with Screenshot Testing and routine-design
Are you afraid the next Pull Request in your React repository, will be the one that causes a serious visual regression? You’ve got plenty of PRs being sent to your GitHub repository every day. You try and review them all closely, but you’ve never felt confident reviewing changes to HTML and CSS. You maintain a high development velocity…but just hope you don’t accidentally break your website.
With routine-design, you can create a safety net against visual regressions. Routine-design is a simple free tool you can immediately start using in your React repository. Routine-design works best on components that have been decoupled from other components. This allows you to test the visual result of HTML/CSS in isolation. Read on to see how to start screenshot testing your components!
Decoupled Components
Say you have a React component. It is basically a JSX file, paired with a CSS file. The JSX file has logic in its render
method. The CSS file has the logic of selecting elements with CSS selectors and applying styling. If there is lots of logic in the render
method, and/or the CSS file, then the component has low cohesion.
If your component had high cohesion, you’d have the following benefits
- Component is simpler, having fewer operations
- The overall component system is more maintainable, because you can change the code in one component without changing the code in other components
- Developers will find the component easier to re-use
In order to achieve high cohesion, you need to decouple your components. Low coupling is often associated with high cohesion. To decouple a component, just extract some of the JSX/CSS logic into new files, creating a new React component.
Smaller, highly cohesive components are easier to screenshot test. The more screenshot testing coverage you have over your component system, the better protected you will be against visual regressions.
routine-design
First things first, install routine-design into your React repository as a developer dependency.
npm install --save-dev routine-design
Next, you’re going to create a “render directory”. This is a new directory of React components. This is not the same as your src
directory. This is a new directory of components, where each component wraps a single component from the src
directory. Each of these component will be captured in a screenshot image. This way, you can take multiple screenshots of the same component, with each screenshot representing a different state. I often create this directory under ./test/render
. Here’s an example screenshot-able React component that is testing the source code for the Logo
component, which I would place in the ./test/render/logo/index.js
file.
import React, { Component } from 'react';import Logo from '../../../src/logo';class LogoTest extends Component { render() { return (<Logo/>); }}export default LogoTest;
Fill out your render directory with as many screenshot-able React components as you want. Next, test that all the components in the render directory actually…render. Add this command to your package.json
.
"scripts": { "render": "routine-design render ./test/render", // other scripts like // "start": "react-scripts start"}
Then run npm run render
. This starts a server on localhost:8080
. The ./test/render/logo/index.js
will render on the localhost:8080/#/logo
URL. Go to the URL associated with each screenshot-able component, and test that everything looks good.
Before we move on to capturing screenshots of these components, let me let you in on a little secret: do not save screenshot images in a repository. Git does not merge images well. Instead, store images on Google Cloud, and keep track of their information in a JSON file. Images hosted on the cloud are easy to look at, and JSON files are easy to merge.
Create a new Google Cloud project if you don’t have one already. You’re going to store the screenshot images in a Google Cloud Storage Bucket. Create a new storage bucket. You only need to fill out the “Name your bucket” step.
In order to make screenshot image review as smooth as possible, I recommend making it possible for anyone to view these screenshot images. Change the permissions for the storage bucket so that allUsers
can view objects in the storage bucket.
I recommend only allowing an authorized service account to upload screenshot images to the storage bucket. Create a new service account. Name the service account, and skip the step for “Grant this service account access to project”. You will specify the service account’s access from the Google Cloud Storage UI in a later step.
And download a JSON key for the service account.
Move the JSON file out of your Downloads directory. I recommend you move it to the same directory as your React repository, and add it to your .gitignore
file. Set your GOOGLE_APPLICATION_CREDENTIALS
environment variable to point to that JSON file.
export GOOGLE_APPLICATION_CREDENTIALS="/home/user/my-app-repo/auth.json"
Then go back to your storage bucket and change so that the service account can create objects in the storage bucket.
That is all you need to connect your developer environment to Google Cloud! But before we can test that you can actually upload screenshot images to Google Cloud…you need to set up one more service: Docker. Docker is a way to run containers, built from images. I’ve created a routine-design Docker image, which packages up code needed to capture a screenshot with routine-design. This containerized software will always run the same, regardless of the infrastructure. Screenshot images captured on a local Mac will be the same as screenshot images captured on a remote Linux environment, like TravisCI.
If you haven’t already, install Docker Desktop on you local developer environment. Make sure you actually start the application. This will make the docker
command available in the terminal. You will be compiling Sass code on this docker image, so build a compatible binding for node-sass. There’s a handy routine-design command for that. Update your package.json
with this new postinstall
command
"scripts": { "postinstall": "docker pull lynnmercier/routine-design && routine-design docker build-node-sass",}
And then run npm install
to download the Docker image. You also need to set up one last environment variable. The ROUTINE_DESIGN_GOOGLE_CREDS
variable will work for both your developer and CI environments.
export ROUTINE_DESIGN_GOOGLE_CREDS=$(cat $GOOGLE_APPLICATION_CREDENTIALS)
Finally, test that you can actually upload screenshot images to Google Cloud. Add this command to your package.json
(note that you will need to specify the GCP project ID, and the name of the storage bucket)
"scripts": { "capture": "routine-design capture ./test/render gcp-project-id my-app-screenshots",}
And run npm run capture
. This process can take more than a minute, so be patient. Once it is finished, run git status
to see which files have changed. You should see some new ./test/render/**/image.json
files. Open the image URLs in the image.json
files to see the screenshot images. It’s actually very important that you open these image URLs and look at the images. You want to build up the habit of checking what the screenshots actually look like. Congratulations, you’ve captured containerized versions of screenshot images! Now with a one-line command you can capture screenshot images of all components in ./test/render
.
Capturing screenshots is great, but we also want to be able to test that existing code matches the screenshots. There’s a handy routine-design command for that. Update your package.json
with this new test:screenshots
command (note that you will need to specify the GCP project ID, and the name of the storage bucket)
"scripts": { "test:screenshots": "routine-design test ./test/render gcp-project-id my-app-screenshots",}
Now run npm run test:screenshots
. Again, this process can take more than a minute, so be patient. You should see a series of passing tests, one for each directory in the render directory. Congratulations! You now have a test suite for asserting local React/Sass code matches the images captured in image.json
! With a one-line command you can test screenshot images of all components in ./test/render
.
If you’re using TravisCI as your CI, I suggest you update your .travis.yml
file to have two separate jobs: one for unit tests and one for screenshot tests. You’ll also need to specify Docker as a service. Note that this code goes below the langauge: node_js
section of your .travis.yml
matrix: include: - script: - npm run test - services: - docker script: - npm run test:screenshots
And finally…add these two to your .gitignore
routine-design-outputauth.json
Before you push all these changes as a PR to GitHub, update TravisCI with the ROUTINE_DESIGN_GOOGLE_CREDS
environment variable. You can use this command echo $ROUTINE_DESIGN_GOOGLE_CREDS
to print out the value. Copy that value into the TravisCI form, quoting it with single quotes.
Now you can push all these changes as a PR to GitHub. TravisCI should kick off a build including screenshot testing. And the tests should pass! If you have any problems with routine-design, be sure to file bugs in the GitHub repository.
Pull Requests
Before you merge your routine-design PR into your code base, you’ll want to instruct your fellow developers how to use the screenshot system. First make sure developers know how to download the auth.json
file: the service account JSON key.
And make sure they set up their environment variables: GOOGLE_APPLICATION_CREDENTIALS
and ROUTINE_DESIGN_GOOGLE_CREDS
. I recommend including all of this in a README
in your routine-design PR.
Developers on your team are responsible for both creating and reviewing PRs. Before they create a PR, they should make sure all tests pass. Make sure developers know to run npm run test:screenshots
, and check that all tests pass, before creating a PR. If there is a failing test, routine-design will give you a convenient Google Cloud URL to help you debug the failure.
When you open the Google Cloud URL, you’ll see subdirectories for each screenshot image that failed within the directory. Each subdirectory has a new.png
and a diff.png
. New.png
is the screenshot of the React/Sass code in the PR. Diff.png
is the diff between new.png
and the image saved in image.json
. Include this information about npm run test:screenshot
in your README.
Of course, your CI will flag any PRs that have failing tests, in case a developer forgets to run npm run test:screenshots
. I highly recommend you set up your GitHub repository such that the CI must pass before a PR can be merged.
With this safety net, you can be truly fearless about merging PRs without causing a visual regression.
Review Visual Changes
Sometimes developers actually want to change how a component looks. This is a visual change, not a visual regression. Say a developer writes some new code, runs npm run test:screenshot
, a test fails, and they open the debug GCP URL. The developer sees that the new.png
image looks good, and the diff.png
makes sense with the new code. At this point, they need to update the golden screenshot image saved in image.json
. This is what the npm run capture
command is for. If the developer runs npm run capture
, routine-design will upload new screenshot images for each changed component, and update the image.json
files.
When the developer creates a PR, they will include any updated image.json
files. If they don’t…the screenshot tests won’t pass and the CI won’t let them merge the PR. The developer reviewing the PR looks at the changes to image.json
as part of the review process. Reviewers can open the image URLs in the image.json
files to see the screenshot images. The reviewer can see both the old and new image URLs in GitHub’s review UI.
Including image.json
files in a PR is basically a way for the PR author to signal to the reviewer that they had to run npm run capture
. It’s clear that the reviewer needs to review visual changes caused by the new code. If the reviewer agrees with the visual changes, then they approve the PR, including the new image.json
files. Once the PR is merged, the new image.json
files become the “gold standard” for future PRs. In this way, the image.json
files are the source of truth for how a component should look visually.
Include this information about npm run capture
in the README in your routine-design PR. I’ve created a simple example PR you can copy. Once you’re happy with the routine-design PR, merge it! Now everyone on your developer team can start benefiting from screenshot testing!