Automatic visual diffing with Puppeteer

So testing, right? We should do it. The thing is, testing is hard, and good testing is reaaaaaaally hard, and tbh I’m pretty bad at testing. So I end up not testing my apps, and then I feel guilty about it, but I’ll stop you now: you can’t run guilt on Travis. If this sounds familiar, then this blog post is for you.

I did a little song-and-dance that sets up Puppeteer* , takes screenshots of your app (like, all the routes you care about), and then compares them to the “golden” ones. If they match, your test passes! Yes, it only works on Chrome. No, it’s not actually unit testing. Yes, it’s literally just counting pixels but you know what? It counts them in both a wide and a narrow viewport size and any testing is better than no testing at all; fight me.

* Puppeteer is an npm library that lets you control Chrome. You know, like a puppet. In particular, Puppeteer makes it super easy to take screenshots (and click on things in your page). It’s like a waaaaaaay less infuriating Selenium, but infinitely harder to spell.

This post looks long because I’ve put all the code I have so that you can copy paste it. Skip to the good part if you already know how to test.

Do the npm

If you want to test things with Puppeteer, you have to setup a thing for the tests, a server that launches your site, and then Puppeteer to look at that site. I have this in my package.json to wrangle these things:

"devDependencies": { 
  "chai": "^4.1.2", 
  "mocha": "^5.0.0", 
  "puppeteer": "^1.0.0", 
  "pixelmatch": "^4.0.2", 
  "polyserve": "^0.23.0" 
}

Explanation:

• I chose Mocha/Chai for testing because that’s what I’m used to. You can literally use any other testing framework you’re comfortable with; I don’t think it matters.
• Pixelmatch is the thing that diffs two images and tells you how many pixels they differ by. It’s super awesome 🏆.
• Polyserve is what I use as a local server. You can use Python or Express or whatever you cool kids use. I’ll point out in the code where it’s Polyserve specific (literally 2 lines), and you can sub in your favourite server there.

Set up your test

In order to tell Puppeteer to investigate your site, you need to:

1. start a test suite
2. that sets up a local server
3. and in each test tells Puppeteer to do something.

My setup looks like this:

Code for setting up a test suite that uses Puppeteer

You can test all sort of things here, by the way. Puppeteer lets you interact
with the page (click on buttons, links, etc), so maybe you want to trigger
different UI states before you screenshot them (like narrow view but also with the navigation drawer opened).

Filing in the blanks

All the heavy lifting (which isn’t very heavy tbh) is done in takeAndCompareScreenshot:

Code for taking a screenshot and comparing it to the golden one

Getting the golden screenshots

This bit is easy. Make a different test suite (just make sure you don’t run it every time you run your tests), and run the page.goto and page.screenshot lines for all the routes you’re testing. I recommend doing the viewport dance too, to get both the wide and narrow screen ones for freeeeee (I am using just the viewport size here, because that’s how my app works. Puppeteer lets yo do device emulation and all sorts of other goodness, so just read the docs). Put all these screenshots in a place; I put mine in a folder called test/screenshots-golden/.

The thing that does the diffing

This is the logic in compareScreenshots, and it’s basically straight out of the Pixelmatch docs:

Code for diffing two images using Pixelmatch

💯 It’s all worth it

Now, when you run your tests (mocha test/ --timeout 5000 in my case), you get something like this:

And if it fails, you get an error and the number of pixels you’re off by.

⭐️

Now go on, navigate to all your routes and test your stuff, and thank me with a photo of your dog.

Originally published at meowni.ca.