Accessibility testing with Quail

Jesse Beach introduces the new accessibility tool that helps developers test their codebases as they build

As the field of web development matures, techniques established by accessibility experts are becoming common practice among user interface designers and developers. Codebases suffer regressions as code complexity increases. We mitigate undesired behaviour changes by describing the expected behaviour in tests.

Too often accessibility criteria is not assessed until the last phase of a project. Fixing a major issue then requires a complete development cycle. As assistive technologies improve, we are building evermore complex patterns using tools like Accessible Rich Internet Applications (ARIA). This code must also be tested. But what should we test? And can we ever say with binary precision that an interface is accessible or not?

I contribute to a tool called Quail that seeks to answer both questions in a free and open codebase. Quail intervenes at the start of the development cycle when developers describe expected behaviours in unit tests. The Quail team is collaborating closely with the Kwaliteitsinstituut Nederlandse Gemeenten to introduce Quail into the Dutch government’s quality assessment toolchain.

Quail’s assessments reflect the best practices prescribed by recommendations like the WCAG. Rather than one-to-one correlations between assessment and recommendation, Quail’s assessments are generic. For example, Quail has an assessment aMustContainText that validates links with text and invalidates links without text. This assessment is one of many used to determine conformance to WCAG Success Criteria 1.1.1 (Nontext Content) and Success Criteria 2.4.4 (Link Purpose In Context). We can run aMustContainText against the rendered templates of an application’s interface components and get a pass/fail result. It does not prove conformance, but it does give us confidence that our applications are moving in that direction.

No automated testing is capable of identifying all accessibility issues. Some aspects, like the constitution of HTML attributes, can be automated. Semantic aspects can at best be partially automated. The Auto-WCAG group — a collaboration of accessibility experts and tool developers — is working to draw these lines clearly.

In the next version of Quail, we will introduce a command line interface, integration with tools like Selenium and extensions for assertion libraries like Chai. Quail is an evolving experiment and your contributions will help us move faster.

Jesse is an accessibility specialist on the Facebook engineering team. She advocates for open source, open standards and information access

This article originally appeared in issue 268 of net magazine.