Using Differencify in Docker and CI

Nima Soroush

6 min readJan 25, 2018

Differencify is a library for visual regression testing.

In this article I am going to show how to use Differencify to test a React App in Docker and CI tools.

This tutorial could be used to tests any other web application (e.g. Angular, Express, PHP, etc …)

I am using React as example. Source code is available here.

Create React App

I am going to use Facebook create react app tool to create a react application and run it in my localhost which I can run visual tests against. To do that run

> npx create-react-app differencify-react
> cd differencify-react
> npm start

All codes are executed using NODE 8.X LTS and NPM 5.X

It will create a directory called `differencify-react` and inside that directory, it will generate the initial project structure and install the transitive dependencies:

differencify-react
├── README.md
├── node_modules
├── package.json
├── .gitignore
├── public
│   └── favicon.ico
│   └── index.html
│   └── manifest.json
└── src
    └── App.css
    └── App.js
    └── App.test.js
    └── index.css
    └── index.js
    └── logo.svg
    └── registerServiceWorker.js

This is what you should see in your terminal after running npm start commands

> npm start

Then open http://localhost:3000/ to see your app. You should see this page

Installing Differencify

You need to add differencify to you package.json to be able to run tests. Run

> npm install differencify --save-dev

This should add differencify as devDependencies in your package.json. You package.json should be like

{
  "name": "differencify-react",
  "version": "0.1.0",
  "private": true,
  "dependencies": {
    "react": "^16.2.0",
    "react-dom": "^16.2.0",
    "react-scripts": "1.0.17"
  },
  "scripts": {
    "start": "react-scripts start",
    "build": "react-scripts build",
    "test": "react-scripts test --env=jsdom",
    "eject": "react-scripts eject"
  },
  "devDependencies": {
    "differencify": "^1.3.0"
  }
}

Test file

Now you need to add test files to your app. Lets create a directory called tests in src directory to accommodate our tests. Now lets create a javascript file called visual.test.js to write our tests.

In this example we will use Jest to run our tests. Differencify has special features for Jest. Inside you test file add

import Differencify from 'differencify';
const differencify = new Differencify({ debug: true });describe('Differencify', () => {
  beforeAll(async () => {
    await differencify
      .launchBrowser({
        args: ['--no-sandbox', '--disable-setuid-sandbox'],
      });
  });afterAll(async () => {
  await differencify.cleanup();
});it('simple', async () => {
    await differencify
      .init()
      .newPage()
      .setViewport({ width: 800, height: 600 })
      .goto('http://localhost:3000/')
      .waitFor(1000)
      .evaluate('(function () { const img = document.querySelector("img"); img.style["animation"] = "unset"; })();')
      .screenshot()
      .toMatchSnapshot()
      .close()
      .end();
  }, 2000);
});

In above example, We will instantiate Differencify and we set the debug flag to true which will output some infos about steps. In beforeAll section we launch a browser which later on will be used to run our tests. We need to pass some browser Arguments to Puppeteer to make it work in docker. You can follow up here. In afterAll section we close all open browsers. Steps are pretty much self explanatory but you can read more about steps API here.

The reason for evaluate that little script in browser is to stop animating React logo before taking screenshot!

Docker

Now it is time to test your application in Docker. To use Differencify in Docker, a docker image been configured specially for this purpose. All you need it to pull Differencify docker image and run you tests on it.

Create aDockerFile inside differencify-react folder and add these lines

FROM nimasoroush/differencify
RUN mkdir ./differencify-react
WORKDIR ./differencify-react
COPY ./package.json ./
RUN npm install
COPY ./src ./src
COPY ./public ./public
CMD ["npm", "start"]

Above will pull Differencify latest image into your docker container and create a `differencify` directory to put your project files into it. No you only need to build docker image

> docker build -t differencify-react .

And

> docker run -it --name differencify-react -v ${PWD}:/differencify/src -p 3000:3000 --rm differencify-react

Now you should be able to open http://localhost:3000/ to see your app again!

RUN TESTS

Now it is time to run your tests against your App which now running in docker. To do this run

> docker exec -it differencify-react npm test

This command will run your visual test and will store an image snapshot in your tests directory

To see the generated image snapshot you need to copy created snapshot from docker machine to your local machine. Run

> docker cp differencify-react:/differencify-react/src/tests/__image_snapshots__/ src/tests/

Now you should be able to see generated image in

/differencify-react/src/tests/__image_snapshots__

And here is the captured image

Now lets make a change to App.js file and rerun the test. Open the App.js and change the <p> tag to

<p className="App-intro">
  Some changes to <code>src/App.js</code>.
</p>

And save the file. Now lets rebuild the docker image and run the app again. Here is what you should see when App runs again

Now lets rerun the test again. Keep in mind that you have your reference screenshot beside your test file now. Here is the result

As you can see the tests failed as we changed some texts. The differencified file been generated just beside your reference snapshot file. To see the differencified file run docker copy command again

> docker cp differencify-react:/differencify-react/src/tests/__image_snapshots__/ src/tests/

Here is the generated file

And diff image

You can easily see the highlighted difference.

No it is time to update our reference screenshot to match our new changes. To do this, just run test command again with update parameter (--updateSnapshot or -u)

docker exec -it differencify-react npm test -- --updateSnapshot

This should update your reference screenshot

And finally you can copy over the updated screenshot

> docker cp differencify-react:/differencify-react/src/tests/__image_snapshots__/ src/tests/

And you will new screenshot in your local machine.

Links

You can find the source code and examples in differencify-react repository. For any question please visit Differencify repo and raise an issue. You can read more…