Automating BrowserStack screenshot generation with CircleCI

Time to catch ’em all. Photo by Melvina Mak on Unsplash

The brief

As part of our continuous integration strategy using CircleCI, we decided to automate our cross-browser testing as part of our CircleCI build workflow. With so much of our traffic — over 90% — coming from mobile devices, BrowserStack’s cloud-based mobile browser testing service was ideal for our needs. So we planned to automatically trigger generation of screenshots from various devices using the BrowserStack API, and save the images as build artifacts in CircleCI.

The process

There are four parts to the process:

1. When a commit to a branch is pushed, the CircleCI workflow is triggered, which builds the branch first, then triggers the script to generate the screenshots based on that branch.

2. The script makes a POST request to browserstack.com/screenshots, which triggers the screenshot generation on BrowserStack’s servers, and returns a JSON object containing a job_id for the next step.

3. Once that job is running, the script makes a GET request to https://www.browserstack.com/screenshots/{job_id}.json, which returns another JSON object containing URLs to the generated screenshots.

4. Finally, the script downloads the images from those URLs into a CircleCI artifacts directory, for us to view once this stage of the workflow is completed.

The code

Dependencies

We’re only using two dependencies for this script: fs for piping the generated images into the artifacts directory, and isomorphic-fetch to handle our HTTP requests.

const fs = require(‘fs’)
const fetch = require(‘isomorphic-fetch’)

Step 1: Start the job and get its ID

const branchName = process.env.CIRCLE_BRANCH === ‘master’ ? ‘staging’ : process.env.CIRCLE_BRANCH;
const username = process.env.BSTACK_USERNAME;
const accessKey = process.env.BSTACK_ACCESS_KEY;
const branchUrl = `YOUR_SITE_URL`;
const basicAuth = Buffer.from(`${username}:${accessKey}`).toString(‘base64’);
const browserList = [ … ]

const getShots = async () => {
  if (!branchUrl) {
    throw new Error(‘No URL set for this branch’);
  }
const response = await fetch(‘https://www.browserstack.com/screenshots', {
    method: ‘POST’,
    body: JSON.stringify({
    url: branchUrl,
    wait_time: 5,
    orientation: ‘portrait’,
    browsers: browserList
  }),
  headers: {
    ‘Content-Type’: ‘application/json’,
    Accept: ‘application/json’,
    Authorization: `Basic ${basicAuth}`
  }
});

const screenShots = await response.json()
  if (screenShots.message === ‘Parallel limit reached’) {
    throw new Error(‘Parallel limit reached! Try again later!’);
  }
const jobID = screenShots.job_id;
}

Using the isomorphic-fetch library and ES6 async/await syntax, our POST request is fairly straightforward. fetch() takes two arguments: the URL we’re posting to (https://www.browserstack.com/screenshots), and an object with method, body and headers attributes. By default, method is set to the string GET, which is why we’ve needed to set it to POST here. headers is another object which tells the server what type of data to expect from us, what type of data we expect to receive back, and our authorisation credentials.

Our body in this request is a stringified object, containing some parameters for the screenshots we want BrowserStack to generate. The browserList array that the browsers property refers to will be an array of JSON objects that each refers to a particular device/browser combination. The BrowserStack Screenshots API docs can tell you how to grab a full list of their available devices.

Depending on your BrowerStack plan, you’ll be limited as to how many parallel tests you can run simultaneously. If you’re trying to exceed that limit, the response you’ll receive from your POST request will only contain a message warning you of this.

At the end of this function, we grab the job_id value from the POST request’s response, and assign it to const jobID, ready for use in step 2.

Step 2: Generate the screenshots

This is the most involved step, so let’s break it down into smaller chunks.

2.1: the GET request

Once again, the fetch() function makes this nice and straightforward:

const getJob = async (id) => {
  const jobResponse = await   fetch(`https://www.browserstack.com/screenshots/${id}.json`, {
    headers: {
      ‘Content-Type’: ‘application/json’,
      Accept: ‘application/json’,
      Authorization: `Basic ${basicAuth}`
    }
  });
  return jobResponse.json();
};

Note that we don’t need to set a method here; it’s GET by default. We also don’t need a body for a GET request, so this time we only need to send the same headers as we did in our POST request.

2.2: monitoring the job’s progress

By now, BrowserStack will have started a job on their end that generates our screenshots. Our getJob() function will return an object that tells us the state of this job. But this job takes a while to finish, and we can’t move past step 2 until state: ‘done’. So we need to keep calling this function, or polling it, until we know that BrowserStack’s job is complete. So we need a pollJob() function:

const pollJob = async () => {
  const job = await getJob(jobID);
  if (job.state !== ‘done’) {
    return await setTimeout(async () => await pollJob(), 3000);
  }
}

pollJob() recursively calls getJob() until job.state: ‘done’. However, as it stands, this doesn’t tell us how the BrowserStack job is progressing, or any errors that might be cropping up. So let’s extend pollJob() to include some logging:

const pollJob = async () => {
  const job = await getJob(jobID)
  const awaitingDevices = job.screenshots.filter(x => x.state !== ‘done’).map(x => x.device);
  if (job.state !== ‘done’) {
    console.log(`\nAwaiting ${awaitingDevices.length} devices:\n${awaitingDevices}`);
    return await setTimeout(async () => await pollJob(), 3000)
  }
}

Now, every time the BrowserStack job is polled, we get a console.log of how many screenshots we’re waiting on, and what devices the pending screenshots are of. The recursion is wrapped in a setTimeout() so that we’re not hammering the job with requests and running the risk of a FetchError. Of course, this might be *too* verbose for your needs, so go ahead and tweak the logs and timings to suit you.

Once job.state === ‘done’, const job is going to be an object containing all the URLs of the screenshot images BrowserStack has so kindly generated for us. Time to catch ’em all.

Step 3: Download the images into CircleCI artifacts

const writeImages = async (imageUrl, filename) => {
  const res = await fetch(imageUrl);
  await new Promise((resolve, reject) => {
    const fileStream = fs.createWriteStream(`/tmp/screenshots/${filename}`);
    res.body.pipe(fileStream);
    res.body.on(‘error’, (err) => {
      reject(err);
      throw new Error(‘ERROR! writeImages failed!’, err);
    });
    fileStream.on(‘finish’, () => {
      resolve();
    });
  });
};
const downloadImages = async (screenshots) => {
  const downloads = await screenshots.forEach(async (shot) => {
  if (shot.state === ‘done’) {
    const urlParts = shot.image_url.split(‘/’);
    const filename = urlParts[urlParts.length — 1];
    await writeImages(shot.image_url, filename);
    console.log(`\nDownloaded ${shot.device}: /tmp/screenshots/${filename}`);
  } else {
    console.log(`\nScreenshot timed out for ${shot.device}`);
  }
});
return downloads;
};

Extend out pollJob() and add an await for it as the final line in the getShots function:

const pollJob = async () => {
  const job = await getJob(jobID);
  const awaitingDevices = job.screenshots.filter(x => x.state !== ‘done’).map(x => x.device);
  if (job.state !== ‘done’) {
    console.log(`\nAwaiting ${awaitingDevices.length} devices:\n${awaitingDevices}`);
    return await setTimeout(async () => await pollJob(), 3000);
  }
  await downloadImages(job.screenshots);
};
await pollJob();

So your entire file will look like:

const fs = require(‘fs’);
const fetch = require(‘isomorphic-fetch’);
const branchName = process.env.CIRCLE_BRANCH === ‘master’ ? ‘staging’ : process.env.CIRCLE_BRANCH;
const username = process.env.BSTACK_USERNAME;
const accessKey = process.env.BSTACK_ACCESS_KEY;
const branchUrl = `YOUR_SITE_URL`;
const basicAuth = Buffer.from(`${username}:${accessKey}`).toString(‘base64’);
const browserList = [ … ];
const getShots = async () => {
  if (!branchUrl) {
  throw new Error(‘No URL set for this branch’);
}
const response = await fetch(‘https://www.browserstack.com/screenshots', {
  method: ‘POST’,
  body: JSON.stringify({
  url: branchUrl,
  wait_time: 5,
  orientation: ‘portrait’,
  browsers: browserList
}),
headers: {
  ‘Content-Type’: ‘application/json’,
  Accept: ‘application/json’,
  Authorization: `Basic ${basicAuth}`
}
});
const screenShots = await response.json();
if (screenShots.message === ‘Parallel limit reached’) {
  throw new Error(‘Parallel limit reached! Try again later!’);
}
const jobID = screenShots.job_id;
const getJob = async (id) => {
  const jobResponse = await fetch(`https://www.browserstack.com/screenshots/${id}.json`, {
    headers: {
      ‘Content-Type’: ‘application/json’,
      Accept: ‘application/json’,
      Authorization: `Basic ${basicAuth}`
    }
  });
return jobResponse.json();
};
const writeImages = async (imageUrl, filename) => {
  const res = await fetch(imageUrl);
  await new Promise((resolve, reject) => {
    const fileStream = fs.createWriteStream(`/tmp/screenshots/${filename}`);
    res.body.pipe(fileStream);
    res.body.on(‘error’, (err) => {
      reject(err);
      throw new Error(‘ERROR! writeImages failed!’, err);
    });
    fileStream.on(‘finish’, () => {
      resolve();
    });
  });
};
const downloadImages = async (screenshots) => {
  const downloads = await screenshots.forEach(async (shot) => {
    if (shot.state === ‘done’) {
      const urlParts = shot.image_url.split(‘/’);
      const filename = urlParts[urlParts.length — 1];
      await writeImages(shot.image_url, filename);
      console.log(`\nDownloaded ${shot.device}: /tmp/screenshots/${filename}`);
    } else {
      console.log(`\nScreenshot timed out for ${shot.device}`);
    }
  });
  return downloads;
};
const pollJob = async () => {
  const job = await getJob(jobID);
  const awaitingDevices = job.screenshots.filter(x => x.state !== ‘done’).map(x => x.device);
  if (job.state !== ‘done’) {
    console.log(`\nAwaiting ${awaitingDevices.length} devices:\n${awaitingDevices}`);
    return await setTimeout(async () => await pollJob(), 3000);
  }
  await downloadImages(job.screenshots);
};
await pollJob();
};
getShots().catch((err) => {
  console.log(err);
});

Step 4: Automate the script in the CircleCI workflow

Time to add the complete script to our npm scripts in package.json:

“build-screenshots”: “node path/to/build-screenshots.js”,

Finally, to ensure that this script runs on every build, we added a browserstack-screenshots job to our CircleCI config.yml…

browserstack-screenshots:
  executor: build
  steps:
    - setup
    - run:
      name: yarn
      command: yarn install
    - run:
      name: build screenshots
      command: |
        mkdir /tmp/screenshots
        yarn run build-screenshots
    - store_artifacts:
      path: /tmp/screenshots

…and inserted it into our workflow after deploying the branch.

Future implementations

With this up and running as part of our CI process, we’re now thinking of ways to extend this functionality to be even more useful to us. Our ideas include posting the screenshots back to the pull request page in Github, or even sending them into our Slack channel once the build process completes.

Having an automated cross-browser testing tool is a huge help to our development process here at LADbible, and we hope we’ll have more to share with you soon!