Automated Accessibility Testing using AXE 4 with Selenium + Java

S Chathuranga Jayasinghe
Apr 9 · 6 min read

Accessibility testing using new Axe 4.0 with Selenium Web driver on Java!

UX means a LOT! Get rid of the headache of UI A11Y testing via automation with Selenium!

ACCESSIBILITY TESTING, in short A11Y Testing(because it has 11 letters between a & y :)) is a separate section that holds a big responsibility.

What is A11Y Testing?

No, it’s not because we don’t have standards. We have several standards that different nations have come up with. So Accessibility testing is a type of software testing performed to make sure an app complies with recommendations of the so called standards. Segment 508 and WCAG rules are the main standards being used in the world.

Section 508 is the availability standard characterized by the US government, to ensure that all US government sites can be gotten to by differently abled people.

Web Content Accessibility Guidelines or WCAG characterize the norms for openness for people, associations, and governments around the world. WCAG 2.0 has been acknowledged as an International Standards Organization (ISO) standard, and numerous nations have embraced WCAG 2.0 as their lawful standard for web openness. (WCAG 2.1 available by now)

Why we need A11Y Testing?

Accessibility is, usability. I think, accessibility often helps not only differently abled people, but also any other people under certain circumstances. For example, the one could watch a video with subtitles when it’s noisy and there are no headphones at a hand. Or, alternatively, when there is no opportunity to read something, you could just listen it through headphones. Applications are often adapted for users with hearing and visual impairments, and, less often, for people who are not able to use a keyboard or any other manual input device.

How to test A11Y?

There are lots of tools to do the a11y testing now. Each tool gives you the capability to pick a standard or several standard to run a test on the app. And it will generate a report as well. So how about doing this manually every time the app changes.. C’mon! Seriously.. Over and over again! Then what about the reports?

Yes, it’s a pain in the head! B’coz you have to be keen on watching for a11y issues manually. That’s why the Automated A11Y testing has come to the lime light. Thank GOD!

Automated A11Y Testing with AXE 4.0

Axe have come a long way since its beginning and the latest version of Axe is 4.1.2 as of 23rd Feb 2021. Deque Labs did major changes when moving forward from Axe 3.0 to 4.0 to ease the use of their library. One major change is, for versions older than 4.0 needed to store the Java Script file used to inject to the page separately as a resource file within the java project, but with 4.0 onwards they have included the Java Script file within the library which has ease out lot of hassle.

Automated accessibility testing is a huge time-saver, it doesn’t require special expertise, and it allows teams to focus expert resources on the accessibility issues that really need them. Unfortunately, most accessibility tools are meant to be run on sites and applications that have reached the end of the development process and often don’t give clear or consistent results, causing frustration and delays just when you thought your product was ready to ship. Axe works with all modern browsers, tools, and testing environments which might be used by a dev team.

Setting up Selenium Test Project for Axe

<dependency>
<groupId>com.deque.html.axe-core</groupId>
<artifactId>selenium</artifactId>
<version>3.1-SNAPSHOT</version>
<scope>test</scope>
</dependency>

That’s it! Now you are good to do some scripting. Before that I’ll walk you through some things you might need to know before.

A11Y Standards on Axe engine

WCAG 2.1

WCAG 2.0

Section 508

Apart from that AXE also has it’s own standards defined like best-practices, experimental and CAT. More information on those can be referred from here.

Axe Commands

AXE.injectAxe

AXE.Builder().analyze()

Let’s write our first selenium-java a11y automation script

I writing my first test to navigate to the amazon.com and check the accessibility issues in their page. The code goes as follows:

@Test
public void testOne(){
driver.get("https://www.amazon.com");
AxeBuilder builder = new AxeBuilder();
Results results = builder.analyze(webDriver);
List<Rule> violations = results.getViolations();
if (violations.size() == 0)
{
Assert.assertTrue(true, "No violations found");
}
else
{
JsonParser jsonParser = new JsonParser();
Gson gson = new GsonBuilder().setPrettyPrinting().create();
AxeReporter.writeResultsToJsonFile(reportFile, results);
JsonElement jsonElement = jsonParser.parse(new FileReader(reportFile + ".json"));
String prettyJson = gson.toJson(jsonElement);
AxeReporter.writeResultsToTextFile(reportFile, prettyJson);
Assert.assertEquals(violations.size(), 0, violations.size() + " violations found");
}
}

This code will automatically inject the JS file and check for the accessibility issues using ALL possible standards on the page. If it finds any issues it will log all the issues to a JSON file and also a text file it the given directory. If you open it, it will list down:

all the issues found

where the issues are found

how to solve the issue

which accessibility standard is violated

and so on..

There are lot of options that we can pass into Builder().analyze() command which will behave in different ways and give different outputs. Check this file for different usages of the command: HERE!

Testing with Specific Accessibility Standards

First add this line of code as a global variable and replace with the accessibility standard you want test. If needed you can specify a specific guideline of a standard as well like wcag241 here.

private static List<String> tags = Arrays.asList("wcag2aa");

And now you can use builder.withTags() to set it up as follows:

@Test
public void testOne(){
driver.get("https://www.amazon.com");
AxeBuilder builder = new AxeBuilder();
builder.withTags(tags);
Results results = builder.analyze(webDriver);
List<Rule> violations = results.getViolations();
if (violations.size() == 0)
{
Assert.assertTrue(true, "No violations found");
}
else
{
JsonParser jsonParser = new JsonParser();
Gson gson = new GsonBuilder().setPrettyPrinting().create();
AxeReporter.writeResultsToJsonFile(reportFile, results);
JsonElement jsonElement = jsonParser.parse(new FileReader(reportFile + ".json"));
String prettyJson = gson.toJson(jsonElement);
AxeReporter.writeResultsToTextFile(reportFile, prettyJson);
Assert.assertEquals(violations.size(), 0, violations.size() + " violations found");
}
}

Have a kick start with this framework code I have developed using Java Selenium with AXE 4.1.2 : GitHub Repository

That’s all! Try it & let me know in the comments section if you have any issues! Happy a11y testing!!!

Geek Culture

Proud to geek out. Follow to join our +1M monthly readers.