In a recent AccessWorld article, Choice Finds from the ATIA 2016 Conference Exhibit Hall, we took a first look at the OrCam from Israeli-based OrCam Technologies. The OrCam is a wearable device that detects what you are looking at, recognizes the presence of text, and speaks the text aloud. The OrCam can also identify various products, US currency, credit cards, and even faces.
While many individuals with visual impairments use smartphones and other mobile devices to accomplish most of these tasks, there is a growing population of people who are newly blind for whom such a device might enable a significant step toward personal independence. People who are newly blind and who do not have substantial technology experience and training may find this device of particular benefit.
At the time of the previous AccessWorld mention, OrCam was not yet available for purchase. It is now available with training via a network of dealers. Recently I had a chance to put the device through its paces in an extended training session. Here's what I found.
A quick Tour of OrCam
The OrCam has two components. The first is the processor unit, which is 5.5 inches by 2.25 inches by .8 inches, and weighs just over 5.5 ounces. It fits easily in the palm of the hand, and is designed to fit into a standard-size hip pocket. A belt clip is also provided, but using a handbag might be problematic. A shoulder bag carried on the right shoulder might work, or a backpack, since there is a 32-inch cable connecting the processor unit to the OrCam. Hopefully, the company will soon be able to offer the option of either Bluetooth or other wireless connectivity.
The processor unit has an audio jack, a charging port, and three buttons: a Power button and Volume Up/Down rocker on one edge, and on the opposite edge the Trigger button, which the user can press to initiate a scan or hold down for two seconds with either of the Volume buttons to open the Settings menu. As you will soon see, however, you can access most essential OrCam features without using the physical Trigger button.
The second component is the camera/speaker head unit, which is connected to the other end of the cable. This component snaps on to the right arm of nearly any standard-size pair of eyeglasses or sunglasses. (Note: wire frame glasses cannot currently be used with the OrCam.) When attached properly, the OrCam 8-megapixel camera is positioned at the very front of the glasses, and the speaker is positioned toward the rear. Weighing in at just over 1 ounce, the head unit does not feel awkward at all to wear. Some prerelease versions of OrCam Eye used bone conduction earphones, but for the release version these have been replaced by a single speaker with sound output directed toward the user's right ear. Those who have hearing deficits in this ear may have a problem using OrCam out of the box. Such users may need to connect their own headphones, external speaker, or hearing aid using the processor unit's audio jack.
The non-swappable battery is rated for 4.5 hours of continuous use. It can be recharged in four hours using a USB power adapter, computer port, or car charger. At the beginning of my two-hour training session the battery was at 100 percent. It ended at 50 percent, so at first glance these numbers do seem fair.
The Two Flavors of OrCam
OrCam comes in two versions: OrCam MyEye, which is priced at $3,500, and OrCam MyReader, which costs $2,500. Let's take these configurations one by one, describing what they can and can't do. Keep in mind that, as is the case with most camera-based accessibility devices and apps, there is a certain learning curve involved. I only had one training session, so I also sought out feedback from an OrCam veteran, which you will find later in this article.
OrCam MyReader
OrCam MyReader focuses on text recognition. There are two ways to instruct the device to identify and speak recognized text. First, look directly at the page or label of text. This can be a bit tricky, at least at first. I tended to look higher and a bit to the right of my intended target. With practice I quickly improved my aim. Now, with the target text directly ahead, press the Trigger button on the processor unit. You will hear a shutter sound as the image is taken. A second or two later the text will begin speaking.
Alternatively, place a finger near the top center of the page or block of text you wish to have read. You will need to place your finger so that the fingernail is facing up. OrCam recognizes the fingernail, and uses it to set the camera's focus. After hearing a confirmation chirp, remove your finger from the text. This action "triggers" OrCam to auto-snap the picture. If you do not hear a chirp, remove your finger from view and then try again.
The developers of OrCam have built their own, proprietary text recognition engine from the ground up. I found the recognition equal to or better than other OCR products, including the KNFB Reader. OrCam prompts you if the text is upside down or too blurry to recognize, and seems to do a good job cleaning up off-center images. It's also quick, and again, there is no data connection needed. You do need adequate lighting, however. OrCam tended to fail in poor light conditions, and there was way to know if the failure was caused by text not being included in the view or because the light was inadequate. This lead to several instances where I kept trying to get the text in view, when my problem was actually poor lighting. Perhaps an inadequate lighting message should be added.
The text is read using either the Ivona, Brian, or Kendra voice. I found the results clear and easy to understand, even with the small built-in speaker. You can change voices, speed, and volume via the Settings menu.
If you wish to stop text reading, simply place your hand in front of the object you're reading. To scan the text: after speech begins, place your finger against the text and slide it down toward where you'd like to skip to. This was a bit tricky, and my attempts resulted in many recognition stops and restarts. With practice I was able to perform the gesture about half the time. I suspect that with further practice I could improve this gesture even more, but I'm not sure it is a feature I would use enough to warrant the effort.
Using OrCam I was able to scan the left page and the right page of a book with high speed and accuracy. OrCam does not save your recognized text after it has been read. You cannot scan ahead in a book as you listen to previous pages, or save your scanned text to a computer or other device. And speaking of computers, when I pointed OrCam at my computer screen it detected I has an Excel spreadsheet open and began reading the data one row at a time.
Taking OrCam to my pantry, I could touch my finger to a can of corned beef hash, for instance, or a box of crackers, and OrCam would speak most visible text. This text tended to be cooking instructions, or nutritional information, from which I could often as not determine what was in the package. Product names were problematic, because they are usually printed in unusual fonts OrCam can't recognize. There is a way to more consistently identify products, which I will describe in the next section. For now, it was time to hit the road.
On the Go with OrCam MyEye
Heading outdoors on the day of my training session I was startled to learn that, even though I have lived in my current house for eight years, I never knew that the street number was printed on top of my garage door. As the instructor and I headed out, the UPS man arrived with two Amazon packages. I was easily able to tell that one of the boxes was for me and one for my wife by simply touching the label with my fingertips, then moving my finger away for the picture to snap. On one of the packages it took me a while to find the label, since it felt so much like the packing tape. Assuming the daylight was sufficient, when OrCam did not chirp I knew I had to pull my finger away and try again.
Corner street signs were also readable, but I cannot say this feature is particularly useful. I had to be in precisely the right spot, and know at which of the four corners the sign was located. Trees and other branches typically interfered, and I had no way to tell if I was even close to being on target.
Heading to a nearby fast food restaurant/gas station/convenience store, I was delighted when OrCam read many of the outdoor signs. I do not think it will be able to distinguish different stores in a strip mall, as most signs use unusual fonts, and OrCam does not do well with lighted neon.
At the order counter, I could pick up snatches of the menu board overhead. I could not read it all, so I had to rely on a printed menu.
By happenstance, the previous occupier of the table I selected had left a newspaper, so between bites of burger I tried reading a few articles. OrCam did an excellent job columizing and only reading down one column at a time. I had to start the text recognition again for the next column. I also needed to have some sense of where things were on the page. OrCam doesn't identify when you're pointing at graphic or picture; it simply doesn't recognize these elements on a target.
When using the Trigger button, as opposed to the point gesture, for reading a newspaper, OrCam reads the text on the entire page, starting at the top-left corner to the bottom-right corner. OrCam announces "reading next text block" to distinguish between headlines and different articles.
Our final stop was inside the convenience store, where I was blown away by OrCam's capabilities. Walking up to a shelf, I touched a finger to an item. It read the item name and the price. It was a bag of dog food. I moved on to another unknown item in a blister pack. OrCam informed me it was a tire gauge, and also provided the price. Last stop—the refrigerator case, where I pulled the door open and began touching beverage bottles. Nearly every time it read what it was and the price tag. It was much easier than my own pantry or refrigerator, since stores always display items with their labels front and center.
OrCam MyEye
Along with all the features described above, OrCam MyEye can recognize currency, credit cards, household or work items you have previously identified and placed into OrCam's memory using voice memos/audio tags, and the faces of friends, family, and coworkers.
Currency must be touched or viewed using the trigger button in order to have it recognized. The various denominations have been pre-added to the OrCam's memory. Other items you will need to add manually.
OrCam MyEye will store and recognize up to 150 credit cards, pantry items, household cleaners, and other objects from the size of a pack of playing cards to a box of cereal. To add an item to the device's memory, press and hold the Trigger button until OrCam prompts you with "Start new product learning. Please point at the product three times at different positions." It's best to take one photo with the item at arm's length, a second closer up, and the third using a different background.
After snapping the photos you are prompted to add an audio name or description of the item. Multiple image sets can be taken, if, say, you want to add the front and back sides of a box, but this will count as two items toward your maximum of 150. Note: You must still have the item in order to clear it from memory.
Now, when you wish to identify that box of teabags or distinguish your sugar canister from your flour canister, touch the item and then pull away your finger, as though you were identifying text. OrCam will speak your audio message.
Identifying faces works similarly to object identification. A person only needs to be photographed and entered into OrCam one time. Press and hold the trigger button for about two seconds. The OrCam device will ask you to please name the person in front of you after the beep. Then you confirm the person by pressing on the trigger button again to complete the facial recognition entry process.
Now, whenever that person comes into view, OrCam MyEye will play your audio tag. Remember, it's playing through a speaker, and though the volume is fairly low, you should probably avoid names such as "My stupid boss," or "The guy who owes me money."
My trainer had pre-entered his facial image into OrCam MyEye. At the fast food restaurant when he wandered away I tried to locate him. It took a while, but finally he came into view and I knew in which direction to look and speak.
One last feature I discovered on my own was when OrCam MyEye abruptly announced, "One person is in front of you." OrCam can recognize that there are up to eight faces in front of you—a handy feature if you're standing in a line, or if you're at a party wondering if you're speaking to a person or a floor lamp.
An OrCam MyEye User Story
As mentioned above, since I only had one OrCam training session I thought it prudent to check in with another, more experienced user. So I spoke with Dorothy Boyd, 80, who lives in Titusville, Florida. Boyd has retinitis pigmentosa, and her vision is limited to four degrees. Her granddaughter is an optician who followed the progress of OrCam as it was being developed and obtained a device for Boyd as soon as one was available.
Boyd uses a speech-enabled iPad and Kindle, but she usually prefers to use her OrCam MyEye to review the screens. "I don't have to worry about accidentally touching something I didn't mean to and changing things," she says.
Boyd also uses her OrCam to read her print Bible and daily devotional. Her husband teaches Sunday school, and she was amazed when she could follow along with his PowerPoint presentations. "OrCam actually helps me orient my scans," she says. "If my finger isn't pointing directly at the PowerPoint display it tells me, 'There's more text to the right.'"
Boyd doesn't have many household items in her object recognition database. She doesn't need to. "I can usually tell what I'm looking at just by the snatches of text I hear," she says. "It's very handy when I'm looking for something in the refrigerator—especially when you have a husband who never puts the mustard back where it belongs."
Boyd does use the facial recognition. Besides her husband, she's also programed her OrCam with images of a number of neighbors and church friends, her two daughters, four grandchildren, and their spouses, along with her four great-grandchildren. "My husband can't sneak up on me anymore," she laughs.
Recently, the family threw a big celebration for Boyd's 80th birthday. "They gave me eight slips of paper with printed messages like, 'Today you're having your nails done,' and 'You're going on a shopping trip with your daughters.'"
Final Thoughts
OrCam MyReader and OrCam MyEye definitely work as advertised. For the newly blind or individuals with physical or cognitive limitations that prevent them from using a touch screen mobile device, the MyReader and MyEye are excellent ways to answer the question: "What is this?" Office workers and others who frequently socialize in groups may also enjoy the OrCam MyEye.
If I have any hesitation about the OrCam it is because of the two different models. Many years ago when IBM started selling printers under the Lexmark name they offered two models of laser printers: a 4-page-per-minute and an 8-page-per-minute. The printers were identical, other than the fact that the company had disabled the higher speed on the low-end model. IBM doesn't sell printers anymore. Similarly, OrCam MyReader and MyEye seem identical other than the preloaded software. MyReader is upgradable to the MyEye, but I think the company would have been better served by setting a single price for the full-featured model. I also wonder how long it will be until someone designs a similar headset unit and connects it wirelessly to an Android smartphone.
Who knows? Perhaps the company is already hard at work on that very thing.
Product Information
Products: OrCam MyReader and OrCam MyEye
Available from: OrCam Technologies
1-800-713-3741 (US)
OrCam is available in English, French, German, and Hebrew in the following countries: United States, Canada, Australia, France, Germany, Austria, Switzerland, UK, and Israel. OrCam will soon be released in Spanish.
Manufacturer's Comments
OrCam's mission is to empower the lives of people who are blind or visually impaired by harnessing artificial vision innovation.
The wearable OrCam assistive technology device provides independence by effectively, discreetly, and instantly reading printed text from any surface, recognizing faces and identifying products and money notes.
Powered by leading minds in the computer vision and machine learning fields, OrCam's team includes dedicated software, computer, and electrical engineers; hardware design experts; and a passionate customer service team—including sighted, low vision and blind members—to provide a visual aid through a discreet, mobile and easy-to-use interface.
Related articles:
- Choice Finds from the ATIA 2016 Conference Exhibit Hall by Jamie Pauls and Shelly Brisbin
- Best Android Apps for People with Low Vision by Shelly Brisbin
More from this author:
