Michele McDonnall, Ph.D., The National Research and Training Center on Blindness and Low Vision
Most job searches today are conducted online and many large employers require candidates to complete their online applications in order to apply for jobs. The problem is that online job applications are often not accessible for screen reader users, based on research conducted between 2002 and 2011 (Bruyère et al., 2005; Lazar et al., 2011, 2012). Given that this research was many years old and web accessibility has received much more attention recently, the National Research and Training Center on Blindness and Low Vision (NRTC) decided to evaluate the current status of the accessibility and usability of online job applications. The results from this study conducted for the NRTC by the American Foundation for the Blind in 2021 (Reuschel et al., 2023) are presented here.
To conduct the study, three experienced, totally blind screen reader users attempted to complete online job applications with 30 Fortune 500 companies. An accessibility engineer observed and recorded data from each of the 90 sessions. Each tester used a different screen reader and browser combination: (1) NVDA version 2020.4 with Chrome version 89 on Windows 10, (2) JAWS 2021 with Chrome version 89 on Windows 10, and (3) VoiceOver with Safari 14 on macOS Big Sur.
The testers identified each accessibility problem and rated its severity from low (could easily be worked around) to critical (prevented the user from continuing with the application). If a critical issue was found, this was documented, and the accessibility engineer helped the tester continue with the application. The testers also rated the usability of each job application on a scale of 1 (no part of the process was usable) to 5 (encountered no barriers at any point).
What We Found
Our major findings about accessibility were:
- 694 accessibility issues were documented during the 90 tests of 30 online job applications, with 73 critical issues found.
- 76.7% (23 of 30) of the online job applications failed (had one or more critical issues) for one or more of the screen reader-browser combinations.
- The different screen reader-browser combinations had very different results across the same online job applications.
- Overall, 50 of the 90 attempts to complete an online job application were successful (55.6% success rate).
- 86.7% (26 of 30) of the online job applications were successful for at least one of the three screen reader-browser combinations.
The overall average usability rating was 3.60 (SD=1.05) out of 5, which corresponds with a rating between “encountered some barriers” and “encountered a few barriers.” This suggests an overall moderate level of usability of the online job applications tested, but ratings for the same application often differed based on the screen reader-browser combination. One job application may have worked well for one screen reader while having a critical issue for another screen reader, and thus a high usability rating for one and a low rating for the other.
The time it took testers to complete the online job applications differed considerably – from 20 minutes to 2 hours and 15 minutes, with an average of 46 minutes. Testers also noted that several companies included features specifically for screen readers, like screen reader-only text or instructions. This was documented by at least 2 of the 3 testers on 40% of the applications. Unfortunately, about a third of these attempts to improve accessibility actually made the experience worse.
How Findings Compare to Older Studies
The study about online job application accessibility conducted in 2011 found that 28.1% of online job application attempts could be completed independently (without sighted assistance) (Lazar et al., 2012). An earlier study found a similar rate of 25% success with corporate job application sites (Bruyère et al., 2005). Our success rate was much higher at 55.6%, considering all 90 tests. Although the three studies used different methodology, this finding may suggest that online job application accessibility has improved for screen reader users. But it’s important to remember that only 23.3% of the 30 applications in this study could be completed independently by all three testers across the different screen readers.
Takeaways
Online job application accessibility may have improved during the 10-year time period of 2011-2021, but there is still a long way to go to make these applications accessible and usable for screen reader users. Several companies tried to make their online applications accessible and usable by screen reader users, as seen in their screen-reader specific text or information, but sometimes those efforts did not work.
For screen reader users who are attempting to complete online job applications, be prepared to potentially spend a large amount of time to complete each application. You should also have the option of sighted assistance available as you go through the process, as it may be difficult or impossible to save your work if you run into a critical accessibility issue.
For companies who have online job applications, it is vital that you test the applications with multiple screen readers – at a minimum the three used in this study. The accessibility and usability issues found in this study, including the critical issues, can easily be resolved by following established Web Content Accessibility Guidelines (WCAG). Techniques for resolving issues such as the ones found in this study are available in the WAI-ARIA Authoring Practices 1.1.
References
- Bruyère, S. M., Erickson, W. E., & VanLooy, S. (2005). Information technology and the workplace: Implications for persons with disabilities. Disability Studies Quarterly, 25(2), 1–16.
- Lazar, J., Olalere, A., & Wentz, B. (2012). Investigating the accessibility and usability of job application web sites for blind users. Journal of Usability Studies, 7(2), 68–87.
- Lazar, J., Wentz, B., Biggers, D., Delair, J., Donnelly, M., Kashim, E., Henin, A., Markakis, J., Matos, A., McNicol, A., Nixon III, J., Osborne, R., Postnova, T., Raja, J., Roberts, R., Serra III, H., Sfakianoudis, V., Tyler, V., & Yun, J. (2011). Societal inclusion: Evaluating the accessibility of job placement and travel web sites. INCLUDE.
- Reuschel, W., McDonnall, M., & Burton, D. (2023). The accessibility and usability of online job applications for screen reader users. Journal of Visual Impairment & Blindness, 117(6), 479–490. https://doi.org/10.1177/0145482X231216757
