Tech Notes
In any conversation about accessibility, there is always discourse and debate about how to implement testing, its different methods, strategies, tricks, and tips – anything you could ever wonder about – with many degrees of effectiveness and success. But a practice that absolutely should not be dismissed is Pair Testing.
What is Pair Testing?
Pair testing is a method in which a screen reader user and sighted individual, whether its a designer, developer, or other member of the team, work together to manually test a website or application to determine the accessibility and usability of the product. The screen reader user typically navigates the interface with their screen reader while the sighted individual follows along to ensure that what the screen reader user experiences matches what is visually presented on the screen. Using this method results in the ability to find issues that would be difficult or impossible for a screen reader user to find and document appropriately to their team, creating the most thorough and efficient results.
Why use Pair Testing?
Sighted and non-sighted users each bring unique perspectives to the table when it comes to identifying accessibility issues in digital products. Sighted users may excel in detecting visual inconsistencies or layout problems that are readily apparent, such as misaligned elements or color contrast issues. On the other hand, non-sighted users, relying on screen readers, are adept at uncovering issues related to HTML structure, alt text for images, or proper heading hierarchy, crucial for navigability. However, both groups inherently miss specific niches of problems relevant to the other. Sighted users might overlook issues that aren’t present on screen, like missing ARIA roles or inadequate keyboard navigation. Conversely, non-sighted users may not notice purely visual issues, such as poorly designed graphics or images that convey important information without alternative text. This is where the collaboration of pair testing proves invaluable as it leverages the strengths of both user groups to create a more comprehensive and inclusive testing approach.
This level of collaboration is particularly beneficial in finding issues related to focus order, content that is visible but not spoken by the screen reader, and when visual appearance of elements do not match the programmatic name, role, or value of that element. All of these issues create accessibility barriers for individuals with disabilities in accessing digital content. As a screen reader user, it can be difficult or impossible to catch many of these issues independently when doing manual testing. For a designer or web engineer, applications may look like they are accessible when observed at surface level, but they may not catch issues without testing from an experienced screen reader user. So pair testing is a good way to double check with each other to ensure that issues are not overlooked accidentally.
It is evident within real-world scenarios that pair testing can be a vital tool for uncovering a diverse range of concerns that may otherwise be overlooked. Sunil Dangwal, a distinguished web accessibility specialist, masterfully demonstrates the pivotal role of this strategy in his experiment testing different variations of accessibility testing. According to “Pair Testing—A Best Practice to Enhance Accessibility Test Coverage,” when accessibility testing was conducted in isolation, a sighted accessibility engineer was able to identify 22% of the issues while a non-sighted tester unearthed 65% of the problems. Yet, when pair testing was brought into the equation, a remarkable transformation unfolded—100% of the issues were identified! This experiment serves as a compelling testament to the comprehensive prowess of pair testing in unearthing accessibility challenges, firmly establishing its indispensable role in ensuring digital inclusion.
Beyond simple efficacy, pair testing also provides a wide range of other benefits to web designers and web engineers. First, working with an authentic assistive technology user gives them a glimpse of some of the everyday challenges that people with disabilities face when digital content is not designed to be accessible for everyone. It can be easier and more effective to show how poorly designed content affects assistive technology users compared to explaining it in a meeting or presentation. Having a firsthand encounter with inclusion roadblocks illustrates the real impact of accessibility problems on assistive technology users, making the problems more tangible for designers and developers.
The second major benefit is that web designers can observe how the assistive technology interacts with their content. This hands-on experience enables them to better understand what modifications or improvements are needed to enhance accessibility. By observing the nuances of this interaction, designers gain a deeper appreciation of the user experience for those with disabilities, allowing them to make more informed decisions to maximize accessibility and usability in their designs.
Pair testing is also highly efficient as it fosters real-time collaboration and problem-solving. Testers can discuss their findings, share observations, and validate their conclusions almost immediately after discovering an issue. This instant feedback loop expedites issue resolution and streamlines the testing process, making it more collaborative and efficient.
And critically, by recognizing the barriers created by inaccessible design through first-hand interaction, web designers and engineers are empowered to become more proficient in crafting accessible content right from the outset. Working with assistive technology users transforms abstract accessibility guidelines into actionable insights, making it easier for designers to identify specific elements that will maximize the usability of their products for individuals with disabilities. As a result, designers can proactively incorporate these principles into their design process, ultimately saving time and resources in the long run. When websites and applications are initially designed with accessibility in mind, it not only saves significant time and effort but also proves to be a wise financial investment for their companies. The budget, time, and resources that would otherwise be allocated to retrofitting and addressing accessibility issues can be conserved and redirected toward other critical aspects of development, ultimately leading to more efficient and cost-effective projects.
Where to Start?
Pair testing is a powerful approach that generates diverse insights benefitting both web designers and developers by providing them with firsthand experience in addressing accessibility challenges while improving the usability of digital products for individuals with disabilities.
If you or your team is just getting started with learning about accessibility, pair testing offers a uniquely effective method of professional development. While we always recommend working with assistive technology users who have experience with testing and providing feedback, it’s also beneficial to begin developing your own knowledge about how assistive technology works and then exploring your product(s) through keyboard navigation, which can help you prepare for a pair testing session with an active assistive technology user.
If you are interested in implementing pair testing into your design and development processes for the benefit of both your users and your team but need some more help getting started, contact us at the AFB Talent Lab to explore how we can collaborate to help you meet those goals.
About AFB Talent Lab
The AFB Talent Lab aims to meet the accessibility needs of the tech industry – and millions of people living with disabilities – through a unique combination of hands-on training, mentorship, and consulting services, created and developed by our own digital inclusion experts. To learn more about our internship and apprenticeship programs or our client services, please visit our website at www.afb.org/talentlab.
