Here at AccessWorld, we have published an ongoing series of articles focused on emerging research and breakthrough technologies to help prevent blindness and restore lost vision. This article describes an emerging solution to a growing accessibility issue: the increasing prevalence of terminals, kiosks, vending machines, and other interactive objects that use touchscreen interfaces inaccessible to people with visual impairments. The article also introduces a brand new "low-tech" solution available right now that can help you better label and navigate the touch controls on your microwave, oven, dishwasher, and other home appliances.
VizLens
If you have used an iPhone for any length of time, you likely have experience with an app called VizWiz. VizWiz was developed by the ROC HCI Group at the University of Rochester, with the support of Google and the National Science Foundation. With VizWiz, you can take a photo and send it along with a question to a contact, Facebook friend, Twitter user, or Amazon Mechanical Turk worker, who can return an answer. Unfortunately, if you've tried using VizWiz lately, you have doubtless noticed a decided lack of response.
"VizWiz was produced as a research project," says Jeffrey Bigham, one of VizWiz's lead developers. "Once other people took the ideas and produced other free and commercial options, it became time to let it lapse."
Bigham's dedication to accessibility has not flagged, however. He's now an Associate Professor at the Carnegie Mellon University Human-Computer Interaction Institute, where one of his PhD students, Anhong Guo, expressed an interest in using computer vision to assist people with visual impairments. Bigham became a project advisor to Guo, as the student began looking for ways to make non-voicing touch interfaces more accessible. Today they have a working prototype, called VizLens.
The VizLens user begins by taking an iPhone photo of a touch interface, and giving it a name (such as "Home Microwave," or "Office Vending Machine"). Then, the user uploads the photo to the service, where the image is crowdsourced for text labeling of each of the interface controls, along with its accompanying position. The image is then sent to a server where it is stored for access whenever the user wants to operate the device.
The user receives a text map of the touch panel, which can be explored using VoiceOver touch and swipe navigation. Say your dishwasher has a single row of touch controls. You would now know that the extreme right button is "Start," and the one to its immediate left is "Rinse and Hold," and so on and so forth. You could also snap a pic of your new TV remote, and then use the resulting map to familiarize yourself with the various controls.
But wait—as they say on TV infomercials—there's more!
After a touch controlled appliance or other device has been mapped, the user can open the VizLens app, activate the named item, then hold the phone with one hand with the camera aimed at the controls, then hover a finger from the other hand over the touchpad. The app announces, button-by-button, which control your finger is pointing at. A second mode allows the user to tap the control they wish to use, at which point the app guides your finger toward the correct position with either beeps or spoken "left, right, up, down" instructions.
"This works well for touch panels with a single layer of controls," says Guo. "But there are many touch interfaces, such as the ones on our office coffee pot and copying machine, that offer dynamic displays, using Mode buttons that change the entire layout with each new press."
The solution the VizLens team has developed involves a one-time series of photographs, one for each mode. "At this point, the app could identify which mode the device is in, and offer navigation for that particular screen," says Guo.
The VizWiz team does not plan to use a library of appliance and other touch panel devices, since the angle and lighting for each user will vary. However, once you have taken the first picture, it will be placed in an on-device library so it won't be necessary to reshoot every time you wish to preheat the oven or start a load of wash.
Future enhancements include using OCR to verify that the information you enter is correct, and combining OCR with crowdsourcing to enable near-real-time use of dynamic displays that have not already been added to your device's library—such as the ticket kiosk at an airport where you have never traveled from before.
"It would also be a simple matter to use the scans to produce 3D templates with tactile controls," notes Bigham.
As to what these tactile controls might conceivably feel like, read on.
HALOS
Currently, the best solution to operating touch interface home appliances is via adhesive dots to mark often used controls or a braille overlay, either handmade or produced by third-party providers or one of a very few appliance makers who offer them. Recently, a new alternative has appeared which offers tremendous potential. It's called the Home Appliance Label and Overlay System (HALOS), produced by Anne DeWitte, proprietor of Tangible Surface Research.
As an engineering grad student, DeWitte studied tactile displays. "Today we have LEDs that can change colors. Hopefully, soon we will also have materials that can shrink or expand using similar display commands to produce touchable buttons, sliders, and even graphics."
It occurred to DeWitte that these same developing screen technologies could also be used to help consumers with visual impairments to operate the touch-sensitive control panels that are becoming standard on more and more home appliances. Thinking ahead toward this possibility, DeWitte began to wonder, "Once we do have the ability, what will the universal tactile representation for, say, a timer clock feel like?"
DeWitte was familiar with braille appliance overlays, but she rejected these as programmable tactile elements. "Not everyone, especially newly blind seniors, reads braille, so they cannot be considered universal design elements."
DeWitte wanted to develop a tactile icon library that identifies tactile shapes that have meaning for the home appliance domain. She took her proposal to Experiment.com, a crowdfunding site for scientific research. Her project received funding, and with the help of design students at the Rochester Institute of Technology—who designed some of the tactile icon shapes—and guidance from members of the Association for the Blind and Visually Impaired (ABVI) in Rochester, who also supplied test volunteer subjects, she began compiling a library of proposed universal tactile icons that could be used over a variety of appliance models and manufacturers.
"The tactile Start button on a Microwave should be the same as the tactile Start button on a dishwasher," observes DeWitte.
Too much research winds up being published in a journal, then put on a back shelf waiting for someone else to turn the theoretical into the actual. But DeWitte is an engineer, and wanted to see tangible results of her work. So she began creating transparent silicon appliance overlays, which used shapes instead of braille to note various functions. If you've used an NLS player you are already familiar with several of these tactile icons: a left-pointing arrow for Rewind, a raised letter X for Stop.
Today, DeWitte offers custom appliance templets on her Etsy store for just $30. Instead of raised dots, these overlays use different shapes to signify what each control does. A raised right-facing triangle shape signifies the Start button on an oven, microwave or a dishwasher, for example, while a popped kernel shape signifies a microwave's Popcorn setting.
"Once I get a product model name and number I can usually find the display layout in a parts list so I can get the layout and measurements," she says. "If it isn't available, I ask the customer to place a quarter against the display for context and then snap a photo for me to work from."
Since the overlays are transparent, a sight-impaired individual can use them without blocking the view for their sighted spouse, children, or other housemates. DeWitte also offers a second option if you don't wish to use a full template. Currently, she produces three different sets of foam-based, stick-on Tactile Icons, which include HALOS for most popular controls. HALOS are priced at $5 per set, but DeWitte is also willing to consider creating custom HALOS packs, or even creating individual custom icons with the shapes of your choice.
"My most popular custom requests are for clothes washers and driers," she says. Which is why DeWitte is actively seeking feedback on what shapes would work best for various appliance functions such as "Delicate Wash," and "Air Dry."
Further Information
You can read more about VizLens and stay up to date on the VizLens website, which includes research materials and a YouTube video demonstration.
HALOS are available through DeWitte's Etsy shop.
Related articles:
- W3C's Web Accessibility Initiative: Leading the Charge in Web Accessibility Across the Globe by John Rempel
- Running the ChromeVox Screen Reader on a Lenovo IdeaPad 100S Chromebook by Bill Holton
More from this author:
