UI testing using Puppeteer and Gitlab CI

As Bill Gates once mentioned, developers should be lazy so that they find an easy solution for a hard task. Does checking of a website with 3 languages and multiple subpages in 8 resolutions sound hard enough for you?

During one of our regular knowledge exchange meetings at Touch4IT, we had a topic of UI testing of web applications. One of my colleagues came with the overview of currently available solutions. Another colleague (Tomáš Tibenský) got interested when he heard about Puppeteer, headless Chrome browser (Chromium actually) from the creators of Chrome and started experimenting. But since Puppeteer has its own myriad of issues, first attempt was unsuccessful. Puppeteer resisted to be installed from Dockerfile, kept crashing, getting stuck. Then I came up with lazy idea — is there any Docker image already on Docker Hub? Of course, it was there.

After some fiddling with Docker image, I managed to launch it from our Gitlab CI after building of “Hello world” application. Just to have two step pipeline, first step of which I nonchalantly skipped after first success by inserting manual launch line. You know, 30 seconds of life wasted during every build.

stages:
 - build
 - test

build:
  stage: build
  image: node:carbon
  when: manual
  script:
   - cd app
   - npm install
  tags:
    - docker

test:
  stage: test
  image: alekzonder/puppeteer:1.5.0-0
  script:
   - cd test
   - yarn install
   - npm start
  artifacts:
    paths:
     - test/screenshots/*/*
    expire_in: 3 days
  tags:
    - docker

To break down this pipeline in gitlab-ci.yml file — build step “build” installs Node application packages and “test” launches Puppeteer UI test written in JavaScript.

“script” contains commands to be executed: “test” is the directory containing our JS file, third-party dependencies are installed by Yarn and then we launch the test itself.

Artifacts are files preserved by CI, in our case screenshots. You can set them to stay alive for a certain time, let’s say 3 days so anybody can see them after the weekend.

“docker” tag launches our Docker-enabled Gitlab Runner.

{
  "name": "hello-test",
  "scripts": {
    "start": "node screenshot-test.js"
  },
  "private": true,
  "dependencies": {
    "chromeless": "^1.5.0",
    "puppeteer": "1.5.0"
  }
}

package.json file above provides us with launch script using “npm start” and includes puppeteer package as a dependency that also downloads and installs Chromium.

Since version 1.7.0, you can also use puppeteer-core package with your own local or remote installation of Chromium.

const puppeteer = require('puppeteer');
const fs = require('fs');
/**
 * Launcher
 */
(async () => {
  //
  // set up Puppeteer
  //
  let browser;
  try {
    browser = await puppeteer.launch({
      args: [
        '--no-sandbox',
        '--disable-setuid-sandbox',
        '--disable-dev-shm-usage'
      ]
    });
  } catch (err) {
    console.log('Error launching browser');
    throw err;
  }

  // launch browser
  try {
    fs.mkdirSync('screenshots/' + name);
    // Web page
    const page = await browser.newPage();
    await page.setViewport({
      width: 1920,
      height: 1080
    });
    await page.goto('https://example.com/', {
      timeout: 10000,
      waitUntil: 'networkidle0'
    });

    // full page screenshot
    const bodyHandle = await page.$('body');
    const boundingBox = await bodyHandle.boundingBox();

    await page.screenshot({
      path: 'screenshots/example-com-fullpage.jpg',
      clip: {
        x: 0,
        y: 0,
        width: parseInt(boundingBox.width),
        height: parseInt(boundingBox.height)
      },
      type: 'jpeg'
    });

    await bodyHandle.dispose();

The simple script we are launching in screenshot-test.js file makes screenshot out of loaded web page.

First, we have to open a browser like in your PC. To prevent memory limits on tall pages, we present “disable-dev-shm-usage” flag.

Next we have to open web page like opening a new browser tab. We have to set target resolution and we are free to navigate to desired page. To be sure it loads properly, we set a wait timeout to 10 000 ms and we wait until there is idle network with zero open connections.

Since we want to make screenshot out of whole web page, we have to hack a little and extract its dimensions and present them as height and width of screenshot. There are some issues with extremely tall web pages but at least it gives you an idea how does it look.

After this, we can freely close the browser. But what if we want to test more web pages or resolutions? Do we have to close pages or even browser? Of course not. But you want to reload page after changing resolution to ensure all the CSS applied is correct.

await page.reload({ timeout: 10000, waitUntil: 'networkidle0' });

What else can you do using Puppeteer? Well, it’s up to you. You have a browser with proper API that can be used from Node.js. Sounds good to you?

Let’s have a look at how you would test status code

const assert = require('assert');
const STATUS_OK = 200;
try {
  let response = await page.goto(url, {
    timeout: 10000,
    waitUntil: 'networkidle0'
  });
} catch (err) {
  return Promise.reject(err);
}
if (!response) {
  return Promise.reject(new Error('Response empty'));
}
assert.equal(response.status(), STATUS_OK, 'Wrong status code');

Maybe you want to test elements on a web page.

// Link hostname only
let result = await page.$eval('a#sk', link => link.hostname);
assert.equal(result, 'example.sk');
// Link full URL
result = await page.$eval('a#sk', link => link.href);
assert.equal(result, 'https://example.sk/link');
// Element content
result = await page.$eval('h1', link => link.innerHTML);
assert.equal(result, 'Main header');

You can use this kind of testing as a part of your TDD or DevOps pipeline. To know more about our usage of tools in DevOps pipeline, read this article ➡

DevOps tools at Touch4IT - Viktor Šulák - Medium