Many AccessWorld readers are familiar with haptic technology, or haptics, but may not be aware of its advancements in the past few years and its now far-reaching benefits. Haptic technology manipulates a user's sense of touch by delivering forces, vibrations, movements, or any combination of those elements in order to simulate a virtual object's size, shape, texture, acceleration, tension, or other features. One example is found in video games. When a user is playing a racing game, some game controllers vibrate to depict the racecar's impact when it crashes.
Haptics technology has now branched to areas like employment and education. People training to become pilots experience haptics in flight simulators for a more lifelike experience of operating an aircraft, such as feeling acceleration during take-off and the jolting of the body during turbulence. Medical school students can perform virtual surgeries, manipulating veins, arteries, organs, and organ systems. Researchers have realized that people with vision loss are great candidates for accessing this technology since visual elements on the computer screen can now be experienced through tactile feedback.
One intriguing haptic project, developed through the creativity and expertise of Dr. Marjorie Darrah, deals with STEM (Science, Technology, Engineering, and Mathematics). Dr. Darrah received her PhD in mathematics from West Virginia State University and has over 13 years experience in teaching higher education. She received funding from the US Department of Education to increase STEM accessibility for people with vision loss through the use of haptics technology. She founded the company eTouchSciences where she and her research team created a series of applications, each one focusing on a science or math concept. The current software is appropriate for elementary and middle school students, but in the future, she would like to cover materials for high school and college students, like a graphing calculator that allows users to input equations and feel their shapes virtually.
For the haptic hardware, Dr. Darrah's team chose the Falcon controller from Novint technologies for its low price tag ($200) and its sturdiness (proven to not break after being used by multiple people for many hours). Within the next few months, these products will be available for sale on the eTouchSciences website. The company is also currently seeking out lesson developers (classroom teachers, professors, and professionals in the visual impairment field) and programmers to make more applications.
Description of the Falcon and How It Works
The Falcon was actually created as a video game controller but works exceptionally well as a classroom tool. It's a futuristic-looking device made of metal and plastic, consisting of a base and three arms connected to a spherical-shaped grip. These arms are positioned around the base at the 12 o'clock, 8 o'clock, and 4 o'clock positions. Thus, when you move the grip, the three arms move simultaneously. Each arm is paired with a motor that updates the arm's position every thousandth of a second, which leads to a high level of accuracy and creates realistic-feeling objects in a virtual environment. It weighs six pounds and is nine by nine by nine inches. It's compatible with Windows 95 or higher, connects to the computer via USB, and has a power supply to plug it into an outlet.
To feel virtual objects, you hold the spherical grip with your dominant hand by positioning all five fingers around its circumference, and then you can move the grip up/down, left/right, and in/out to sense all three dimensions of objects. It's like a Cartesian graph found in mathematics where you deal with the x, y, and z planes.
Feeling virtual objects with the Falcon differs from how you use your sense of touch in reality. With real-life touch, you have the entire surface area of your hand(s) placed on the object to discern texture, size, and shape. Yet, on the Falcon, you feel a small patch (less than a square inch in size) through the grip at once, so you have to move the grip around to piece these patches together to discern the whole object. One analogy is to picture extending two fingertips to explore the surface area of an apple, finding the top portion, moving to the bottom, and then going around the circumference.
The Falcon can depict virtual objects that are sizes of four by four by four inches or smaller. For reference, a virtual bowling ball is about four inches in circumference, and a virtual pea is about one-eighth inch in circumference. Two pounds is the maximum virtual weight of an object, so this amount is used when showing the weight of the heaviest items. Then, lighter ones are scaled to this number.
Select this link to view a short video showing how the Falcon is used with the learning applications.
Documentation and Installation of Software
Currently, no instruction manuals have been written for the applications, but they are very easy to operate, requiring you to memorize five to ten keystrokes that are familiar to users of Windows and assistive technology. There is an accessible tutorial application, though, that summarizes directions well. Since the Falcon is a mainstream device that is usually used in gaming, the packaged instructions with it are in regular size print. It would be wise for eTouchSciences to offer documentation for all its products in accessible formats.
Before working with the applications, you must install the drivers. The installation process is standard, requiring you to navigate through various dialog boxes with the "Back" and "Next" buttons. One or two installations do require the use of mouse cursors (like the one that requires you select the "Yes" checkbox after reading the license agreement), but anyone with an intermediate understanding of a screen reader will be fine. Obviously, the actual applications' installations are 100 percent accessible.
General Features of All Applications
While there is discussion of creating a version for the iPhone and other Apple devices, the software currently requires Windows 95 or higher and the use of a sound card. All applications have built-in speech using voices from Natural Reader. Many people (including myself) who have evaluated the applications would prefer using the screen reading software we are comfortable with, which would allow us to pause speech, read by character/word, read by sentence, spell words, and adjust voice rate/pitch. Currently, none of these features are present in the applications. Another recommendation is to have all spoken messages be displayed in text on the screen for those with usable vision or for sighted teachers to follow along easier. One positive aspect of each application is that they all contain high resolution graphics that use high-contrast color schemes. Every application starts with a home screen where an introduction to the lesson is given, and all the operation keystrokes are summarized. Successive screens cover the lesson content.
Three-Dimensional Shapes
The Three Dimensional Shape application is the only one that does not contain a lesson. Rather, it's an interactive tutorial that helps you practice feeling the shape, size, and texture of different objects so that you are able to utilize the Falcon to its fullest capacity. After going through the standard home screen with spoken instructions, you navigate through screen after screen, each one showing common objects (nine in total), including a soccer ball, a can of soup, and a block of wood. Each screen has a spoken message that says, "Examine this golf ball with the haptic device. Can you feel its shape? Can you feel its texture?" Three textures are covered: smooth, bumpy, and sandpaper. Cylinder, sphere, cube, and rectangular prism are geometric solids found throughout. Each object contains both elements, such as a bumpy golf ball. You can test your identification accuracy by pressing "Enter" on each screen for a quiz.
One potential improvement for this tutorial is for there to be a way users can access detailed information about each object, such as its orientation on the computer screen and its virtual size. Some of the testers at AFB found it difficult to determine this information because every screen has a single object in the center with a large amount of open space encompassing it. One obvious solution is to make every item bigger to fill up the unused areas. Also, spoken direction could be more detailed and say, for example,
"Pull the grip all the way out on the haptic device, so the arms are fully extended. Then, start pushing in until you reach the center of the screen where the front of the object, a block of wood, is located. Its virtual size is two inches in length, one inch in width, and one inch in height."
Surface Area and Volume of a Cube
This application is a good starting point for new users since it requires the most rudimentary skills in discerning objects using the Falcon. The speech output on the first screen gives the definitions of surface area and volume. The next set of screens discusses how their formulas are derived from the measurements of the length of a side of a cube. The last few screens present these figures in various sizes, and you are able to take quizzes in which you perform the actual calculations to ensure you grasp these concepts.
Gravity on Planets
The goal of this application is to have students understand the difference between weight and mass due to the amount of gravity present. The first screen's speech output gives definitions for these terms and shows a bowling ball of a set mass. Successive screens show this same bowling ball, but it is located in different celestial bodies' atmospheres, including Earth, Earth's moon, Jupiter, Mars, and Venus. The awesome thing is that you can pick up the bowling ball to compare differences in weights! This virtual sensation is caused by change in tension of the Falcon's arms. When you are touching the ball surface, there is no resistance coming through the arms to the grip, but when you press and hold down a button on the grip, you instantly feel tension increase in the arms. You can even change its location on the screen by picking it up and moving your hand on the Falcon's grip to go from the foreground to background.
Caption: Using the Falcon and the Gravity on Planets application to explore Mars's gravity
Exploring the Atom
This application utilizes the Falcon's push/pull or attraction/repulsion forces to discuss concepts of atomic particles. The speech output on the first screen gives definitions of the three kinds of atomic particles (proton, neutron, and electron), and they are shown as spheres lined up in a row for you to compare. So, when you move the grip toward the location of the neutron, which has no charge, there will be no resistance encountered to reach its surface. However, when you move near the positively charged proton, you must apply force to touch it because it repels, and when you move toward the electron, you are pulled towards it automatically. The grip moves on its own to get to its surface to mimic how electrons attract. Successive screens show diagrams of various elements, each with electrons orbiting in a ring around its nucleus of protons and neutrons. Each of these element screens has a quiz that tests concepts related to atomic weight, atomic number, and balancing of charges.
The Bottom Line
Readers may be pondering: Is using haptic technology more beneficial than using traditional tactile diagrams? The answer: yes. The Falcon is much less expensive than all the tactile graphics embossers currently on the market, which cost thousands of dollars. Many people with vision loss also have difficulty discerning 3D representations of objects that are drawn on a 2D surface as shading is used to represent shadow and position, and length of lines are used to show perspective. Because people are used to feeling how objects appear in reality, the Falcon can do just that, creating the most lifelike 3D replica. Additionally, haptic technology's virtual simulation allows students to experience much more of the world that was previously restricted to them for reasons like danger, microscopic size, or no realistic means of reaching the actual destination. It's easy to use haptics to visit the ocean floor, look inside a volcano, see the geography of Antarctica, feel parts of a cell, touch a virus, travel through the galaxy, or feel the anatomy of a poisonous snake.
The full potential of haptics related to education of people with vision loss has not yet been realized, so spread the word! Readers should explore the eTouchSciences website to learn more. Dr. Darrah encourages anyone interested in getting involved, whether it is through submitting a lesson plan to be made into an application or helping with software development, to contact her or eTouchSciences.
Product Information
Vendor: eTouchSciences
Science Applications Pricing: Please contact vendor for pricing details.
Falcon 3D Controller Price: $200.00
Contact: Dr. Marjorie Darrah
E-mail: mdarrah@inforesearchcorp.com
Falcon 3D Controller Manufacturer
