Tech Notes
An automated scanner is a special piece of software which is mainly designed to scan and test a single web page or an entire website to determine whether or not a web application is compliant with the Web Content Accessibility Guidelines (WCAG) or Section 508 accessibility regulations.
Often companies use these tools to quickly gauge compliance with the accessibility guidelines to make their website accessible. This is clearly a mistake.
Automated scanners do have some uses. Spot checking, for example. Don’t want to check if you have labeled all the button controls on each and every page on your website? Test the site through an automated scanner, and it will check it all for you very quickly. You can also check the presence of items like link text, UI control labels, alternative text, basic color contrast, page title, document language, and form labels through an automated scanner.
However, these scanners only allow us to check only the presence of these items and not the validity of the content. Legal compliance, and more importantly usability, demand much more than that. It requires that your website be accessible to and used by assistive technology users, and this clearly cannot be proved by an automated scanner.
Therefore, we must ask this question: can we fully rely on these automated tools to make a web application 100% accessible? And the answer to that is a resounding “no.”
Because any automated scanner is a piece of software, it doesn't have very much in the way of common sense. It will interpret each and every accessibility guideline literally, without bearing any other thought to what else is on the page.
Here are some of the areas in which automated scanning often falls short and one must perform manual testing to get the most accurate results.
Keyboard Accessibility: You always want to determine if your site is optimized for keyboard-only functionality, such as by moving between sections of a web page, accessing all menus, skipping to content links, and establishing focus on links, menus, tabs, and form fields. This is not easy to test using a scanner and is at the root of true user accessibility. (Read some more of our thoughts about how to easily test keyboard usage.)
Alternative Text for Images: If an automated scanner reports that all images contain alt text, then the tool will report a pass for this guideline. But what if the alt text isn't actually descriptive of its image? What if alt text is full of nonsensical keywords? An automated scanner can’t possibly know this information.
Content Structure: A website may be perfectly coded and conform to the highest coding standards, which is what an automated scanner evaluates. However, if its content is poorly structured, the site will prove difficult to impossible for some if not most assistive technology users.
Use of Color: The automated scanner has no way of knowing if a website is using color as the only visual means to convey information or not.
Form Labels: Automated scanning tools can check whether the form field labels are present, but they cannot determine whether that label accurately describes the purpose of the form field.
Form Error Messages: The form error messages should be meaningful and keyboard accessible. The automated scanner can check if the form error messages are present but cannot check whether they are clear and keyboard accessible.
Consistent Navigation: The web pages should be predictable. If navigation is consistent from page to page within a website, this makes it much easier for screen reader users, screen magnification users, and others to find particular items within the navigation which an automated scanner can't do. Therefore, logical flow or logical meaning of content can only be assured through manual testing.
Distinguishable Link Text: Automated scanners can check whether link text is provided, but they cannot determine whether that text accurately describes the link’s purpose. This is one of the primary reasons automated testing fails.
Color Contrast: You want to increase usability through design by making it easier for users to see and hear content, including separating foreground from background. Some scanners get this right most of the time, but there are many edge cases that need to be tested manually.
Yes, automated scanning tools are most organization's first choice for scanning their site for accessibility violations, which on the surface makes sense as most websites are very large and fairly complex to comb through by hand. But are these tools a little too good to be true? Can you really test the accessibility of a website so easily? No, you can’t because solely relying on automated scanners for every part of the testing process to make the website accessible is impossible.
Ultimately, the best option for testing website accessibility is to not rely only on automated scanners but at minimum combine that effort with manual testing, preferably with real assistive technology users to accurately check, validate, and remediate the website accessibility.
You can explore our resources for digital inclusion by following this link: https://www.afb.org/consulting.
