Automated Testing is NOT Enough for Web Accessibility ADA Compliance

Over 80% of WCAG 2.0 Success Criteria requires Manual Review and 100% of the new WCAG 2.1 Success Criteria will require the same.

By Giacomo Petri and Christian Federici

The debate around what level of accessibility validation can be achieved with automated testing tools has been on-going for 20 years — with a range of different opinions. The simple fact is automatic accessibility testing alone is not enough; and the WCAG 2.1 update has increased the importance of conducting manual accessibility review and testing with assistive technology to ensure full conformance.

Below is a handy chart that breaks down WCAG 2.0 and 2.1 Success Criteria identifying if Automatic testing can be used, or if Manual Review is required. The numbers tell the hard truth that manual review or testing with assistive technology is fundamental. That means more effort than most companies want to hear and more effort than most accessibility industry advocates want to admit, as it is placing a large burden on companies.

With that said, Automatic testing is a great starting point and an important part of continued on-going accessibility; and any test that can be automated should be for efficiency. But why?

Automatic testing can pick up a lot of issues, fast, maybe even the majority of issues in quantity, when looking to achieve WCAG 2.0 or 2.1 AA compliance. It is also the most popular way for law firms or industry advocates to try and identify sites not doing well. So, testing and passing automatic tests will improve the accessibility of a site and reduces the potential for legal demand letters but does not ensure conformance with WCAG or guarantee an assistive user can use the site.

Where does Automatic Accessibility Testing fall short?

WCAG 2.0 and 2.1 establishes the guidelines and suggests the techniques to test pages through a number of Success Criteria at each level of conformance (A, AA, or AAA). An automated test can only be used for a small number of these Success Criteria, leaving the majority of criteria requiring a manual review. Many automatic testing tools try to overstate their completeness by stating how many automatic tests it can run, but rarely do they mention the number of Success Criteria that can NOT be fully tested automatically leading to misconceptions and incomplete results.

Please click here to view our checklist chart — Source: UsableNet Audit Team 2018. *AQA is UsableNet’s Automated Accessibility Platform.

So how can you test your site fully to ensure compliance?

The short answer, conduct both automated and manual testing against all WCAG 2.0 and 2.1 Success Criteria. However, this can become a costly endeavor when needing to perform tests at a regular cadence as part of multiple releases. Therefore, having an accessibility platform, such as UsableNet AQA, can help to efficiently streamline the entire accessibility testing process involving automated, manual review and user testing.

UsableNet AQA provides automated testing features that can be added to the development process, such as a Chrome extension and API, along with the expert review features that Developers and QA teams can use simultaneously to check all Success Criteria which will need manual inspection and documentation. Additionally, conducting user testing with users of Assistive technology will significantly reduce this manual review work. Unique to the UsableNet AQA platform, a user tester can provide feedback and usability results directly next to the specific code review. UsableNet AQA is fully accessible and all functions of testing, review and user testing input can be performed by all users.

In summary, while Automated accessibility testing is a good and recommended part of an accessibility validation process, they will remain insufficient to achieving and maintaining a fully compliant site. To ensure your site and user flows are accessible and usable to people of a wide range of abilities begin with the automatic test and following that with manual screen reader verification from either experienced accessibility reviewers or daily users of Assistive technology.

For a how-to of accessibility integration continue reading, Integrating Accessibility into your Development Life Cycle.

Originally Posted Here: http://blog.usablenet.com/automated-testing-is-not-enough-for-web-accessibility-ada-compliance

Follow us on Twitter: https://twitter.com/Usablenet