Using Storybook as a Visual Testing Platform

My experience with Visual Testing and how you can implement it in your projects

Ema Suriano
Apr 28, 2019 · 8 min read

I’ve always had visual testing on my to do list, until a few weeks ago when I finally decided to mark it as done.

In this article, I’m going to share my experience working with this strategy of testing (which does not replace the others), and the integration with my current development tool: Storybook.

In case you don’t know what Storybook is, this is the definition they provide on the official website:

Storybook is an open source tool for developing UI components in isolation for React, Vue, and Angular. It makes building stunning UIs organized and efficient.

Image for post
Image for post
Storybook Demo

Regarding visual testing, it’s a testing approach that consists of taking real pictures, and then comparing them with the previous version. This comparison is made by comparing both pictures and checking if the pixels match.

In case you already understand the concept of Snapshot testing; this is the same. The difference is that you compare a picture, and not some resulting code.

In case you’re thinking, “How are these two concepts connected?”, Storybook insists on developing your components in isolation, which is the best scenario for visual testing, in order to avoid unwanted differences in our images.

One way of thinking about this approach is that every story defined in Storybook is a visual test in our application.

One last thing before checking the implementation — there are excellent services out there that I tried and they work flawlessly — like Percy, Applitool, and many more.

In this article, I want to show you the handmade way of implementing a visual testing workflow, which has its own cons and pros regarding the mentioned services.

Let’s start with a fresh project, by bootstrapping it with create-react-app and installing Storybook using its CLI.

$ npx create-react-app visual-testing-with-storybook
$ cd visual-testing-with-storybook
$ npx -p @storybook/cli sb init

When the installation is finished, you can check if everything was set up properly by running yarn storybook, and you should see the homepage with two stories that were created by default.

Now, let’s create a simple component for testing. I decided to create a simple button called DuplicationButton, so that, every time the user clicks on it, the children provided by props will be duplicated.

Not very useful, but it will serve as a great example.

Let’s add some stories for the component.

And this is how it looks in Storybook.

Image for post
Image for post
Demo DuplicationButton stories

Generating Tests Based on Stories

In order to implement one test per story, there is a really helpful add-on in Storybook, called storyshots.

In order to install it, you need to run:

$ yarn add -D @storybook/addon-storyshots react-test-renderer

Then create a test file, where you initialize storyshots. Let’s name it storyshots.test.js .

// src/storyshots.test.js
initStoryshots from '@storybook/addon-storyshots';
initStoryshots({ /* configuration options */ });

In order to run it, execute yarn test. Now for each story, there is a test with a snapshot, where you can check what the output (the component rendered) of the story is.

These snapshots are going to be automatically generated every time we run our tests, and in case they have a difference with the previous one the test will fail.

Image for post
Image for post
Test result with storyshots

Getting Visual

In the code above, the function initStoryshots accepts a configuration object with an option called test, which allows us to change the comparison method for each story/test.

Luckily, there is an add-on called storyshot-puppeteer, which, as the name suggests, creates an instance of a browser, navigates to the story, takes a picture and compares it with the previous one.

To install it:

$ yarn add -D @storybook/addon-storyshots-puppeteer

Then, inside the storyshots file, you need to override the test comparison with imageSnapshot from the puppeteer add-on. It’s also needed to specify the URL where storybook will be running, in order to know where to navigate.

One thing to mention, is that now our tests depend on having an instance of Storybook running. I recommend managing two terminals at the same time for this kind of scenario. In one terminal you run yarn storybook, and in the other yarn test.

The old snapshots are obsolete (you can safely delete them), because now we have a new folder called __image_snapshots__, where there’s a picture for each story. And every time a component changes what it renders, tests will fail and you can check it with a visual diffing, between the stored image snapshot and the new image.

In the following diffing, I changed the border-color of the button from pink to blue. The original image will be on the left, the new on the right, and in the middle will be the difference between both, with the changes in red color.

Image for post
Image for post
Visual diffing in action

Running on CI

Having two terminals open at the same time is something we can only do in development. But when it comes to automating this task, things can be a little tricky.

Luckily, someone thought about this problem and made an npm package, called start-server-and-test, which does exactly that.

You need to specify the command to start the server, and once the server is up and running, it will use the command test that you specified. Once that process is done, it will kill the server.

yarn add start-server-and-test

Inside package.json, you need to create a new script which will start storybook, then listen until http://localhost:9009 is up and running, and execute the tests.

"scripts": {
"test": "react-scripts test --coverage",
"storybook": "start-storybook -p 9009 -s public",
"test:ci": "start-server-and-test storybook http://localhost:9009 test",

Interactive Visual Testing

Sometimes you might want to interact with the story before taking the picture.

In order to do that, you need to follow a different approach than the one previously showed. You can achieve this by using a customer jest matcher called jest-image-snapshot, and any end-to-end framework. In this case, I picked puppeteer.

yarn add -D jest-image-snapshot puppeteer

In order to enable jest-image-snapshot, you need to extend the function expect from jest.

The recommended way of achieving this with create-react-app, is by creating a file inside src with the name of setupTests.js. This file will be loaded before all the tests are started, which will enable us to use this custom matcher.

// src/setupTests.js
import { toMatchImageSnapshot } from 'jest-image-snapshot';
expect.extend({ toMatchImageSnapshot });

Create the test file where we’re going to check the behavior of DuplicationButton, let’s check how it looks when the user clicks on it twice.

In order to run this test, we need to start Storybook, or you can use yarn test:ci which will do it for you. This is what the screenshot looks like:

Image for post
Image for post
Interactive visual testing

My Experience Working With Visual Testing

As with all the new technologies/frameworks, I tried visual testing first during a side project, in which I saw a possible improvement.

The project itself was a collection of weather icons made with React and styled-components, called weather-styled-icon.

Image for post
Image for post
weather-styled-icon showcase

In the first release of this library, I wrote all the tests with Enzyme, following a structural testing strategy.

In simple words, I was rendering an icon with mount and then running checks to see if a node exists. In order to check how it looked, I was running expect.toMatchSnapshot of the resulting styles.

As you can imagine, this way of testing is very time consuming, but nevertheless, I was able to finish all of them.

The problem came when I decided to update the version of styled-components from v3 to v4, because I wanted to start using some of the cool new APIs, like ThemeProvider, or styled APIs to style existing styled-component components.

After I made all the changes inside the code, all my tests were broken, due to moving, adding and removing components. I also changed the internal implementation of almost all of them.

At this point, I realized I was testing the implementation of my components, and not the real output I was expecting.

So I made the decision to give visual testing a try, because it seemed like the most suitable testing strategy for my situation — where I want to check exactly how the icons and the variations look like.

I followed the same steps that I already explained above, and I ended up having a better suite of tests with way less code. This is the difference of lines of the merged Pull Request.

Image for post
Image for post
Visual testing lines diff weather-styled-icon

Do’s and Don’ts

I think that visual testing is an excellent way of testing, and I highly encourage you to at least try it in a side project, where you can play with it and see if you like it or not.

However, I would like to highlight a few important things to do and not to do regarding this topic:

  • ❌ Don’t check the styles of your components with code. Instead, take a real image of the component.
  • ✅ In case you’re using Storybook in your development workflow, you can have one visual test for all the stories, without any real effort.
  • ❌ Visual testing does not replace other strategies of testing. It just adds another testing layer to your application.
  • ✅ You can easily integrate it with your current end-to-end tests.

Last words 👋

One more thing before you leave, I decided to start a newsletter so in case you want to hear about what I’m posting please consider following it! No SPAM, no hiring, no application marketing, just tech posts 👌

Better Programming

Advice for programmers.

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app