Be Fearless with Screenshot Testing and routine-design

Are you afraid the next Pull Request in your React repository, will be the one that causes a serious visual regression? You’ve got plenty of PRs being sent to your GitHub repository every day. You try and review them all closely, but you’ve never felt confident reviewing changes to HTML and CSS. You maintain a high development velocity…but just hope you don’t accidentally break your website.

With routine-design, you can create a safety net against visual regressions. Routine-design is a simple free tool you can immediately start using in your React repository. Routine-design works best on components that have been decoupled from other components. This allows you to test the visual result of HTML/CSS in isolation. Read on to see how to start screenshot testing your components!

Photo by Brooke Lark on Unsplash

Decoupled Components

Say you have a React component. It is basically a JSX file, paired with a CSS file. The JSX file has logic in its render method. The CSS file has the logic of selecting elements with CSS selectors and applying styling. If there is lots of logic in the render method, and/or the CSS file, then the component has low cohesion.

If your component had high cohesion, you’d have the following benefits

• Component is simpler, having fewer operations
• The overall component system is more maintainable, because you can change the code in one component without changing the code in other components
• Developers will find the component easier to re-use

In order to achieve high cohesion, you need to decouple your components. Low coupling is often associated with high cohesion. To decouple a component, just extract some of the JSX/CSS logic into new files, creating a new React component.

Smaller, highly cohesive components are easier to screenshot test. The more screenshot testing coverage you have over your component system, the better protected you will be against visual regressions.

routine-design

First things first, install routine-design into your React repository as a developer dependency.

npm install --save-dev routine-design

Next, you’re going to create a “render directory”. This is a new directory of React components. This is not the same as your src directory. This is a new directory of components, where each component wraps a single component from the src directory. Each of these component will be captured in a screenshot image. This way, you can take multiple screenshots of the same component, with each screenshot representing a different state. I often create this directory under ./test/render. Here’s an example screenshot-able React component that is testing the source code for the Logo component, which I would place in the ./test/render/logo/index.js file.

import React, { Component } from 'react';
import Logo from '../../../src/logo';
class LogoTest extends Component {
  render() {
    return (<Logo/>);
  }
}
export default LogoTest;

Fill out your render directory with as many screenshot-able React components as you want. Next, test that all the components in the render directory actually…render. Add this command to your package.json.

"scripts": {
  "render": "routine-design render ./test/render",
  // other scripts like
  // "start": "react-scripts start"
}

Then run npm run render. This starts a server on localhost:8080. The ./test/render/logo/index.js will render on the localhost:8080/#/logo URL. Go to the URL associated with each screenshot-able component, and test that everything looks good.

Before we move on to capturing screenshots of these components, let me let you in on a little secret: do not save screenshot images in a repository. Git does not merge images well. Instead, store images on Google Cloud, and keep track of their information in a JSON file. Images hosted on the cloud are easy to look at, and JSON files are easy to merge.

Create a new Google Cloud project if you don’t have one already. You’re going to store the screenshot images in a Google Cloud Storage Bucket. Create a new storage bucket. You only need to fill out the “Name your bucket” step.

UI for creating a Google Cloud Storage bucket with name my-app-screenshots

In order to make screenshot image review as smooth as possible, I recommend making it possible for anyone to view these screenshot images. Change the permissions for the storage bucket so that allUsers can view objects in the storage bucket.

UI for adding allUsers as a Storage Object Viewer

I recommend only allowing an authorized service account to upload screenshot images to the storage bucket. Create a new service account. Name the service account, and skip the step for “Grant this service account access to project”. You will specify the service account’s access from the Google Cloud Storage UI in a later step.

UI for creating a Google Cloud service account called my-app-screenshots

And download a JSON key for the service account.

UI for downloading a JSON key for the GCP service account

Move the JSON file out of your Downloads directory. I recommend you move it to the same directory as your React repository, and add it to your .gitignore file. Set your GOOGLE_APPLICATION_CREDENTIALS environment variable to point to that JSON file.

export GOOGLE_APPLICATION_CREDENTIALS="/home/user/my-app-repo/auth.json"

Then go back to your storage bucket and change so that the service account can create objects in the storage bucket.

UI for adding service account as a Storage Object Creator

That is all you need to connect your developer environment to Google Cloud! But before we can test that you can actually upload screenshot images to Google Cloud…you need to set up one more service: Docker. Docker is a way to run containers, built from images. I’ve created a routine-design Docker image, which packages up code needed to capture a screenshot with routine-design. This containerized software will always run the same, regardless of the infrastructure. Screenshot images captured on a local Mac will be the same as screenshot images captured on a remote Linux environment, like TravisCI.

If you haven’t already, install Docker Desktop on you local developer environment. Make sure you actually start the application. This will make the docker command available in the terminal. You will be compiling Sass code on this docker image, so build a compatible binding for node-sass. There’s a handy routine-design command for that. Update your package.json with this new postinstall command

"scripts": {
  "postinstall": "docker pull lynnmercier/routine-design && routine-design docker build-node-sass",
}

And then run npm install to download the Docker image. You also need to set up one last environment variable. The ROUTINE_DESIGN_GOOGLE_CREDS variable will work for both your developer and CI environments.

export ROUTINE_DESIGN_GOOGLE_CREDS=$(cat $GOOGLE_APPLICATION_CREDENTIALS)

Finally, test that you can actually upload screenshot images to Google Cloud. Add this command to your package.json (note that you will need to specify the GCP project ID, and the name of the storage bucket)

"scripts": {
  "capture": "routine-design capture ./test/render gcp-project-id my-app-screenshots",
}

And run npm run capture. This process can take more than a minute, so be patient. Once it is finished, run git status to see which files have changed. You should see some new ./test/render/**/image.json files. Open the image URLs in the image.json files to see the screenshot images. It’s actually very important that you open these image URLs and look at the images. You want to build up the habit of checking what the screenshots actually look like. Congratulations, you’ve captured containerized versions of screenshot images! Now with a one-line command you can capture screenshot images of all components in ./test/render.

Capturing screenshots is great, but we also want to be able to test that existing code matches the screenshots. There’s a handy routine-design command for that. Update your package.json with this new test:screenshots command (note that you will need to specify the GCP project ID, and the name of the storage bucket)

"scripts": {
  "test:screenshots": "routine-design test ./test/render gcp-project-id my-app-screenshots",
}

Now run npm run test:screenshots. Again, this process can take more than a minute, so be patient. You should see a series of passing tests, one for each directory in the render directory. Congratulations! You now have a test suite for asserting local React/Sass code matches the images captured in image.json! With a one-line command you can test screenshot images of all components in ./test/render.

If you’re using TravisCI as your CI, I suggest you update your .travis.yml file to have two separate jobs: one for unit tests and one for screenshot tests. You’ll also need to specify Docker as a service. Note that this code goes below the langauge: node_js section of your .travis.yml

matrix:
  include:
    - script:
      - npm run test
    - services:
      - docker
      script:
      - npm run test:screenshots

And finally…add these two to your .gitignore

routine-design-output
auth.json

Before you push all these changes as a PR to GitHub, update TravisCI with the ROUTINE_DESIGN_GOOGLE_CREDS environment variable. You can use this command echo $ROUTINE_DESIGN_GOOGLE_CREDS to print out the value. Copy that value into the TravisCI form, quoting it with single quotes.

TravisCI UI for adding environment variables

Now you can push all these changes as a PR to GitHub. TravisCI should kick off a build including screenshot testing. And the tests should pass! If you have any problems with routine-design, be sure to file bugs in the GitHub repository.

Pull Requests

Before you merge your routine-design PR into your code base, you’ll want to instruct your fellow developers how to use the screenshot system. First make sure developers know how to download the auth.json file: the service account JSON key.

UI for creating a key for a GCP service account

And make sure they set up their environment variables: GOOGLE_APPLICATION_CREDENTIALS and ROUTINE_DESIGN_GOOGLE_CREDS. I recommend including all of this in a README in your routine-design PR.

Developers on your team are responsible for both creating and reviewing PRs. Before they create a PR, they should make sure all tests pass. Make sure developers know to run npm run test:screenshots, and check that all tests pass, before creating a PR. If there is a failing test, routine-design will give you a convenient Google Cloud URL to help you debug the failure.

Console output of a failing screenshot test, including a Google Cloud URL

When you open the Google Cloud URL, you’ll see subdirectories for each screenshot image that failed within the directory. Each subdirectory has a new.png and a diff.png. New.png is the screenshot of the React/Sass code in the PR. Diff.png is the diff between new.png and the image saved in image.json. Include this information about npm run test:screenshot in your README.

Of course, your CI will flag any PRs that have failing tests, in case a developer forgets to run npm run test:screenshots. I highly recommend you set up your GitHub repository such that the CI must pass before a PR can be merged.

UI for updating GitHub to require TravisCI to pass before merging a PR

With this safety net, you can be truly fearless about merging PRs without causing a visual regression.

Review Visual Changes

Sometimes developers actually want to change how a component looks. This is a visual change, not a visual regression. Say a developer writes some new code, runs npm run test:screenshot, a test fails, and they open the debug GCP URL. The developer sees that the new.png image looks good, and the diff.png makes sense with the new code. At this point, they need to update the golden screenshot image saved in image.json. This is what the npm run capture command is for. If the developer runs npm run capture, routine-design will upload new screenshot images for each changed component, and update the image.json files.

When the developer creates a PR, they will include any updated image.json files. If they don’t…the screenshot tests won’t pass and the CI won’t let them merge the PR. The developer reviewing the PR looks at the changes to image.json as part of the review process. Reviewers can open the image URLs in the image.json files to see the screenshot images. The reviewer can see both the old and new image URLs in GitHub’s review UI.

Including image.json files in a PR is basically a way for the PR author to signal to the reviewer that they had to run npm run capture. It’s clear that the reviewer needs to review visual changes caused by the new code. If the reviewer agrees with the visual changes, then they approve the PR, including the new image.json files. Once the PR is merged, the new image.json files become the “gold standard” for future PRs. In this way, the image.json files are the source of truth for how a component should look visually.

Include this information about npm run capture in the README in your routine-design PR. I’ve created a simple example PR you can copy. Once you’re happy with the routine-design PR, merge it! Now everyone on your developer team can start benefiting from screenshot testing!