How to Automate Web Accessibility Testing

Source: aum

About Pa11y

Pa11y (pronounced “pally”) is a set of free and open source tools that aim to help designers and developers make accessible websites. They offer a range of utilities, including a dashboard interface, web service, and desktop application. To learn more about Pa11y, visit the homepage here.

Five steps to integrate Pa11y CI

I made a sample app to try out and demonstrate the functionalities of Pa11y CI. You can find it on my Github page and follow along. While this demo is built with the Angular 7 framework, Pa11y CI can be used with any websites.

  • http-server —serves static files from the command line; we use this to serve the sitemap
  • wait-on —waits for files, ports, sockets, and http(s) resources to become available; we use this to hold off on running pa11y against our website until our website finishes building and the sitemap is served
  • npm-run-all — runs multiple npm scripts in a parallel or sequential fashion
  • husky — sets up git hooks
$ npm install -g pa11y-ci
$ npm install http-server -g
$ npm install wait-on
$ npm install npm-run-all --save-dev
$ npm install husky --save-dev
  • level : The level of issue to fail on (exit with code 2); options include error, warning, and notice.
  • timeout : Time in milliseconds that a test should be allowed to run before calling back with a timeout error; defaults to 30,000. Note that this is the timeout for the entire test run, including the time to initialize Chrome, load the page, and run the tests.
  • threshold : The number of errors, warnings, or notices permitted before failing the test with exit code 2.
  • ignore : Array of codes and types to ignore; defaults to an empty array. Use the HTML Code Sniffer guidelines to determine the exact names of which regulations to ignore.
  • viewport : This can be used to check different window sizes, mobile views, etc.
  • action : User interactions simulated before Pa11y tests the page; it defaults to an empty array. There are a number of options for this configuration, including the ability to click on an element, set a field value, check or uncheck a field, take a screen capture, wait for an element to load, wait for a fragment, path, or URL to load, as well as navigate to a URL. Take a look at the official documentation to see the options and how to use them.
  • includeNotice and includeWarnings : Both of these are boolean values. Issues with a type of notice or warning are not directly actionable, so they are excluded by default, but Pally provides the option to include them via the .pa11yci file.
  • concurency : The number of tests that should be run in parallel; defaults to 2.
  • useIncognitoBrowserContext : Run test with an isolated incognito browser context, stops cookies being shared and modified between tests; defaults to false.
// .pa11yci{
"standard": "WCAG2AAA",
"level": "error",
"defaults": {
"timeout": 5000,
"threshold": 2
"ignore": [
" WCAG2AA.Principle1.Guideline1_4.1_4_3.G18.Fail"
"urls": [
"url": "http://localhost:4200",
"screenCapture": "pa11y-screenshots/home.png"
"timeout": 8000,
"url": "http://localhost:4200/login",
"screenCapture": "pa11y-screenshots/login.png",
"actions": [
"wait for element #login-form to be visible"
// package.json"scripts": {
“sitemap”: “http-server src/assets/nuts-n-bolts_sitemap.xml”,
“pa11y-ci:sitemap”: “wait-on http-get://localhost:4200 && wait-on http-get://localhost:8080 && pa11y-ci --sitemap http://localhost:8080",
“a11y-check”: “npm-run-all -p start sitemap pa11y-ci:sitemap"
  • pa11y-ci:sitemap : runs Pa11y CI tests after localhost:4200 is done loading.
  • a11y-check : runs the above two scripts in parallel.
// package.json"husky": {
"hooks": {
"pre-push": "yarn a11y-check"


The internet is arguably the most powerful tool in the modern world, offering unprecedented access to information and thereby opportunities. Bob Kahn, co-inventor of the internet protocols (IP), once said:

What the internet is going to be in the future is what society makes it.

I hope to be part of an open and inviting web community, and I believe it starts with us — web designers and developers — making sure our content is accessible to everyone. The tools covered above are a solid start for preventing a11y errors early in the development process and making applications more inclusive. But adopting web accessibility is a continuous, group effort.



Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store



Software engineer with a passion for human-focused technology. Let’s connect: