Accessibility Regression With Protractor in Less Than 30 Minutes

Published in

Capital One Tech

6 min readMay 6, 2020

black and white classic style alarm clock with 2 bells

Preface

Most UI developers understand the importance of accessibility in today’s day and age. If you don’t — brush up! Inaccessible websites not only create problems for their users, but can also put the producing company at risk. Having been tasked with maintaining the accessibility of a few of Capital One’s products, I’ve gone through the struggle of building, changing, and maintaining several UI accessibility regression suites. In this article I’m going to show you how to build an accessibility regression suite quickly using Protractor, Cucumber, and the open source tool axe-webdriverjs (a wrapper of the open source axe-core tooling).

The best part — you’ll be able to copy and paste your way to success if you follow along.

Prerequisites:

1. Node (I used v12.13.1)

2. IDE (I use VSCode)

3. Java installed on the machine

4. Javascript / Typescript working knowledge

5. Familiarity with Protractor and Cucumber frameworks

Create the Project

Open a terminal in the directory you’ll place your project in. First, let’s create and initialize our project:

mkdir accessibility-regression
cd accessibility-regression
npm init — yes

Make the appropriate directories to hold your feature files, step definitions, and reports:

mkdir -p reports src/features src/steps

At this point, your project should look something like this:

|- accessibility-regression    |- reports    |- src        |- features        |- steps    |- package.json

Install Dependencies

Now, let’s install all of our needed dependencies up front. In the terminal, run the following commands:

##
# Installs:
# Selenium webdriver accessibility tool wrapper
# Angular end to end testing framework
##
npm install — save-dev protractor
npm install — save-dev axe-webdriverjs##
# Installs:
# Cucumber Framework
# Protractor plugin to enable cucumber js
# Chai assertion library
# Cucumber HTML reporter
##npm install — save-dev cucumber
npm install — save-dev protractor-cucumber-framework
npm install — save-dev chai
npm install — save-dev cucumber-html-reporter
##
# Installs:
# Typescript packages that will enable Typescript in e2e source code
##npm install — save-dev typescript
npm install — save-dev ts-node
npm install — save-dev @types/node

Configure the Project

Let’s start working on setting up the Protractor test suite. In the project’s base directory, place the following configuration file with the name ‘tsconfig.json’. This file configures ts-node to interpret or transpile TypeScript source code.

{
    “compilerOptions”: {
        “target”: “es6”,
        “module”: “commonjs”,
        “moduleResolution”: “node”,
        “inlineSourceMap”: true,
        “declaration”: false,
        “noImplicitAny”: false
    },
    “exclude”: [
        “node_modules”,
        “plugins.ts”
    ]
}

Protractor requires a configuration file for basic functionality. It will dictate browser settings, frameworks used, feature and step definition file locations, reporting configurations, and more. You can see that we’ve configured these options in the code below. Inside the ‘src’ directory of your project, place the following code in a configuration file named ‘protractor.conf.js’:

// Use this — interprets .ts as .js
require(‘ts-node’).register({ project: ‘./tsconfig.json’ });// Needed for cucumber html reporting
const reporter = require(‘cucumber-html-reporter’);exports.config = {
    // ‘Custom’ as we’re using the protractor version of cucumber
    framework: ‘custom’,    // path relative to the current config file
    frameworkPath: require.resolve(‘protractor-cucumber-framework’),    // Require feature files
    specs: [
        ‘./features/**/*.feature’, // accepts a glob
    ],    // Run chrome browser in headless mode, disable gpu as it
    // doesn’t help in our case
    capabilities: {
        browserName: ‘chrome’,
        chromeOptions: {
            args: [‘ — headless’, ‘ — disable-gpu’],
        },
    },    cucumberOpts: {
        // This is where the results are stored
        format: [‘json:./reports/results.json’],        // Requires these files on test launch
        require: [‘./steps/*.ts’],
        tags: true,
    },    // When the tests are finished running
    afterLaunch() {
        // Configure reporting options
        const options = {
            columnLayout: 1,
            theme: ‘bootstrap’,
            jsonFile: ‘./reports/results.json’,
            output: ‘./reports/cucumber_report.html’,
            reportSuiteAsScenarios: true,
            scenarioTimestamp: true,
            launchReport: true,
        };        // Generate the report
        reporter.generate(options);
     },
};

Create Feature Files

At this point, we’re ready to build our feature files. A feature file describes scenarios in which the test is described in plain text that can be understood by both coders and non-coders. A scenario is made of steps in a “Given, When, Then” order. Steps are also easily readable step descriptions, and are matched to the code in our step definition file via regex expressions. To save time, we’ll build these steps with the knowledge that protractor is essentially made to work with angular sites — but what if we aren’t testing one? We’ll separate this out into two different steps, as Protractor will throw errors when navigating to a non-Angular page by default.

Our intent is the following:

Navigate to url {url}
Test for accessibility violations

Given this information, create the two feature files below in the ‘src/features/’ directory. We’ll use Angular’s site for our Angular implementation, and Google for our non-Angular implementation.

angular.feature:

Feature: Angular Accessibility Test
    Scenario: Test for accessibility violations on Angular.IO
        Given Navigate to page https://angular.io
        Then The page should be accessible

google.feature:

Feature: Google Accessibility Test
    Scenario: Test for accessibility violations on Google
        Given Navigate to non-angular page https://google.com
        Then The page should be accessible

Create Step Definitions

Now we’ll implement our steps so that our tests can actually run. Navigate to the src/steps/ directory and create a file named 'stepDefinitions.ts’. Place the following code into the file.

import { After, Given, Then } from ‘cucumber’;
import { browser } from ‘protractor’;
import { expect } from ‘chai’;// For accessibility
const axe = require(‘axe-webdriverjs’);// Navigate to angular page
Given(/^Navigate to page (.*)$/, async function (url) {
    await browser.get(url);
});// Navigate to non-angular page
Given(/^Navigate to non-angular page (.*)$/, async function (url) {
    browser.ignoreSynchronization = true;
    await browser.get(url);
});/** Cleanup */
After(function (scenario) {
    browser.ignoreSynchronization = false;
});

You can see that the first step definition is utilizing Protractor’s default navigation. The second utilizes the ignoreSynchronization setting, which allows us to navigate to non-Angular pages without error, as the browser is not listening for Angular specific events to consider navigation complete. After each step, we reset ignoreSynchronization to its default setting.

Now all that’s left is the step definition for testing the page for accessibility violations. Axe-Core is the open source tooling being utilized at the core of our package. The project has a huge following which makes it great for reliability, maintainability, and feature suggestions. In ‘stepDefinitions.ts’, add the following step definition before the ‘After()’ function used for cleanup. This step will analyze the page for accessibility violations. The ‘withTags’ function takes an array of rulesets. For this example, we’ll use wcag21aa (wcag2.1aa).

/**
* Accessibility step — https://github.com/dequelabs/axe-webdriverjs
*/
Then(/^The page should be accessible$/, async function() {
    /**
    * Available tags: wcag2a, wcag2aa, wcag21a, wcag21aa,
    * section508, best-practice, experimental, cat.
    */
    let results = await axe(browser.driver)
        .withTags([‘wcag21aa’,’cat’])
        .analyze();    /**
    * Check the results for violations. If there are any, indicate
    * what they are.
    */
    expect(results.violations.length).to.be.equal(
        0,
        `Accessibility Violations Found: ${JSON.stringify(
        results.violations,
        null,
        2
        )}`
    );
});

Run the Regression

Now we’re ready to run! By default, Protractor comes with the ability to spin up a Selenium server using webdriver-manager. Some configuration options have been added to allow for some flexibility in the sites you may be testing. For more information on its use, enter ‘webdriver-manager –help’ in the terminal.

Download the selenium server jar and chromedriver binaries. To do this, in your terminal, type:

node ./node_modules/protractor/bin/webdriver-manager update — standalone=true — gecko=false — ignore_ssl

Run the suite from the project’s base directory:

npx protractor src/protractor.conf.js

The results will be viewable under reports/cucumber_report.html. The page should auto-launch when the tests are complete. Note — proxy/vpn may interfere with this step. If this step times out, it’s a good place to start debugging.

Analyze the Reports

The generated reports will be color coded (green for success, red for failure), provide overview statistics, and allow you to drill down into failures and successes. Failures will provide detailed messages and locators, allowing you to pinpoint problem areas and work to fix them.

grey dashboard with black text and 2 green pie charts

That’s it! You now have a fully functional accessibility regression suite. Add feature files for your pages and always remember to keep your suite up to date. The suite can be run locally, or integrated into your CI tools. Stay accessible, friends!

References:

DISCLOSURE STATEMENT: © 2020 Capital One. Opinions are those of the individual author. Unless noted otherwise in this post, Capital One is not affiliated with, nor endorsed by, any of the companies mentioned. All trademarks and other intellectual property used or displayed are property of their respective owners.