Using Pa11y CI and Drone as accessibility testing gatekeepers

Dominic Fraser
Jul 16, 2018 · 8 min read
Pa11y and Drone working together

Here at Pa11y, we think making the web more accessible improves it for everyone.

Pa11y provide a range of automated tools, that can help to spot common accessibility issues detectible within HTML. Set up once, it can then be ran at will, helpful for determining the current state of your product, and in making sure no issues creep in as it develops further. It provides a great baseline, and with descriptive error messages helps to quickly fix issues that would otherwise build up over time.

a CI-centric accessibility test runner, built using Pa11y.

It works through a descriptive JSON config file, where multiple URLs to be tested can be specified, and has full support for Pa11y’s Actions to mock user interaction with a page. It is designed to be usable within a continuous integration pipeline, and is very quick to get up and running.

Setting up Pa11y CI

Let’s walk through an example installing Pa11y CI, setting up a config file, and running some tests both locally and in Drone. It’s assumed you already have a package.json, if not run npm init and follow the prompts.

Installing Pa11y CI

First we need to install Pa11y CI, adding it to our devDependencies as we will not be using it in production. We do this using:

Setting up the test config

.pa11yci.json specifies the full configuration of how Pa11y CI will run, from the URLs to test, the standard to test against, rules to ignore, and user actions to mock.

"standard": "WCAG2AAA",
"level": "error",
"defaults": {
"timeout": 20000,
"wait": 2000,
"ignore": []
"ignore": [
"urls": [
"url": "",
"viewport": { "width": 320, "height": 480 },
"actions": [
"wait for element .site-brand to be visible",
"screen capture screenshots-output/mobile-main-view.png",
"click element .site-nav__item button",
"screen capture screenshots-output/mobile-expand-menu.png"
"headers": {
"Accept": "text/html"


If there are no errors you can expect a result like this:

Both URLs passing successfully
The error displayed in detail on a failed test

Using Pa11y CI during a Continuous Integration pipeline

Pa11y CI is designed with Continuous Integration in mind. We will look at using it with Drone, but there are also examples of using it with Travis CI.

"chromeLaunchConfig": {
"args": ["--no-sandbox"]
"wait-on": "wait-on http-get://localhost:3000",
"component:ci": "npm run component --hotReloading=false --watch=false"
image: // a Docker image that contains chrome headless is required
group: tests
- npm run component:ci &
- npm run wait-on
- npm run test:accessibility

Final Thoughts

Pa11y CI is a powerful open source tool that is actively maintained and updated, helping to make products more inclusive for all users.


how hackers start their afternoons.

Dominic Fraser

Written by

Edinburgh, Scotland.

how hackers start their afternoons.