J.J. Meddaugh
TalkBack has been Google's built-in screen reader since nearly the beginning of Android, and is available on any Android phone purchased today. And while Android has a devoted fan base of screen-reader users, including the writer of this post, some major features have been missing from TalkBack, especially when compared to VoiceOver on the iPhone.
Version 9.1, released in late February, is a major step toward closing this inequity, with the highlight being support for multi-finger gestures, those commands that can be activated using two, three or four fingers. Below we'll talk about what's new and who can take advantage of these features.
First, a Bit of History
As most readers likely know, smartphone users who turn on a screen reader often interact with their devices using gestures. When a screen reader user taps on the screen, the text under their finger is spoken, but nothing is activated or opened. Typically, tapping twice on the screen is what actually opens or interacts with the spoken item. This is an example of a gesture, and the foundation for commands on VoiceOver or TalkBack.
On Android, these gestures have been mostly limited to using just one finger, which severely limits the number of motions that can be made to execute commands. Swiping right or left would move to the previous or next item, for instance, but if you wanted to perform a more complex action from a menu, you needed to use what have been commonly termed "angle gestures." These gestures are performed by moving your finger in one direction, and then moving it in another direction, all without lifting your finger. To open the TalkBack menu, you would need to draw an L on the screen, or swipe down, then right. It was very important to distinctly draw both portions of the L separately, or TalkBack would often not recognize it.
Meanwhile, those using Samsung phones had access to an entirely different screen reader called Voice Assistant. Voice Assistant included some additional commands not available in TalkBack, but often would cause frustration for users who ended up with two screen readers on their phones.
If this all sounds a bit confusing to you, you are among the chorus of users begging for something more user friendly. With only single-finger gestures available, the number of possible commands was naturally limited.
A New TalkBack, for Some
Version 9.1 of TalkBack is perhaps the biggest change in the screen reader since the introduction of gestures in 2012. Most of the angled gestures have been replaced with multi-finger equivalents that will be easier for many, but you do need a supported modern Android phone to get these features.
TalkBack 9.1 is available on Google Pixel phones, the flagship devices released directly by the company. These are typically the first phones to gain any new Android features. In addition, Samsung and Google have collaborated on this version of TalkBack, so modern Samsung devices such as Galaxy phones now also work with this version of TalkBack, instead of using their own proprietary screen reader. Other devices that are likely to gain the features from this new version include phones in the Android One program, which Google certifies to receive the latest software updates as they are released. These include selected phones from Nokia, LG, Motorola, and Xiaomi.
If you have a supported device with automatic updates turned on or have recently updated the apps on your phone, you should have received the update by now. TalkBack 9.1 rolled out to most users of Pixel and Samsung devices at the beginning of March. One way to check to see if you have the new update is to tap once with three fingers on the screen. If you have the new update, the TalkBack menu will be displayed and spoken.
Getting to Know the New Gestures
For seasoned TalkBack users, many of the existing gestures remain intact. In fact, if you are one of the users who liked the original angle gestures, they are still available for your use. But there are now simpler ways to perform many actions, and a variety of new commands as well. We'll focus on the default gestures for each command, but be aware that any of these gestures can be customized to suit your needs.
First, there is now a new way to navigate between elements. Swiping right or left will now always move between each element on the screen, regardless of navigation setting. This is your baseline level of navigation. If you wish to move by character, word, or link for example, you do this by first selecting the element you wish to navigate by swiping right or left with three fingers, and then swiping up or down to move by that element.
This gives us an opportunity to mention another new multi-finger gesture. To start reading at the current item, and continuously read items until the end of the screen or until you stop TalkBack from speaking, you can perform a triple tap with two fingers. TalkBack will move through each element one at a time. You can swipe right or left while it is reading to more quickly move forward or backward through the elements.
One of the most popular gestures on VoiceOver is what is commonly called the Magic Tap, which is tapping twice with two fingers on the screen. TalkBack now has a similar version of this command, which will start or stop media playback, or answer and hang up a phone call. You can also still turn on the feature to end a phone call by pressing the power button, which is an accessibility option that is independent of the screen reader.
As mentioned above, there is now a much simpler way to locate the TalkBack menu, by tapping once with three fingers. The TalkBack menu is where you will find a variety of advanced and lesser-used commands including reading or spelling the last spoken phrase, showing or hiding the screen, and changing the spoken language. While some of these commands also have assigned gestures, others do not. But you can assign just about any command to a gesture by opening TalkBack Settings from the TalkBack Menu and selecting Customize Gestures.
New Commands for Editing Text
TalkBack has always included a way to copy the last spoken text to the clipboard, but new commands make text editing a much simpler chore. As an example, you may want to do this to copy the contents of a note to a text message or email.
To select some text, first locate a box with editable text, such as a text message or note. Turn on text selection mode by double tapping and then holding with two fingers. In other words, you will tap on the screen twice using two fingers, but hold that second tap for about a second. TalkBack will speak "Selection Mode On" to let you know you have correctly performed the command. Then, swipe up or down with one finger to select or deselect text. TalkBack will speak each letter, word, or line as you pass it and let you know if it has been selected or deselected. Remember you can swipe right or left with three fingers to change the navigation mode between characters, words, or lines. When you have selected the desired text, perform the same two finger double tap and hold gesture. TalkBack will turn off selection mode and speak the text that was selected. If you were to swipe left or right, or otherwise move your focus away from the editable text, the selection mode will be turned off and text will not be selected.
Now that you have text selected, you can use one of the new three-finger gestures to work with this text. These include:
- Three-finger double tap: Copy the selected text to the clipboard
- Three-finger double tap and hold: Cut the selected text to the clipboard
- Three-finger triple tap: Paste the contents of the clipboard at the current cursor position
Remember, you can still also copy the last spoken text to the clipboard. This is now easily done by activating the TalkBack menu by tapping once with three fingers and then selecting the Copy Last Spoken Phrase item.
Voice Commands
TalkBack can now be operated using a variety of voice commands. By default, this feature is available either through the TalkBack menu, or by swiping right and then up with one finger. Admittedly, if this is a command you plan to use regularly, you may want to assign it to an easier gesture from TalkBack settings.
You can use a host of voice commands to move quickly to items on the screen, read text, and control TalkBack features. For instance, you can say the name of an item on the screen and TalkBack will move focus to that item. You can say "speak faster" or "speak slower" to control the rate of speech. You can say "first" or "last" to move to the first or last item on the screen, or "select all," "copy," or "paste" to manipulate text. Say "help" to get a complete list of commands.
Getting Help
With all of these new commands, there's now more to learn. Thankfully TalkBack has added a new command to enable a gesture learning mode. Tap once with four fingers, and then try any of the gestures mentioned above, or other combinations of single, double, or triple tapping with one to four fingers. To exit this mode, swipe right until you hear the Finish button, and then double tap.
The TalkBack tutorial has also been updated to reflect these new features. To start the tutorial, tap twice with four fingers.
In addition, Google has created a page with all of the TalkBack commands.
Feature Requests
TalkBack 9.1 is a huge step forward for Android accessibility, but like just about any app, there are always places that could be improved. For instance, having a quick way to start and stop text dictation, whether using the current two finger double tap gesture or another gesture, would make text input easier. You can use the voice command feature mentioned above, and then say “type bla bla bla” to type text, or locate the Voice Input button on the on-screen keyboard, but a dedicated gesture would still be helpful for many.
There are several gestures that are currently duplicated as the transition is made from single-finger to multi-finger commands, so hopefully some of these can be replaced with unique commands to broaden the number of functions that can be performed without the need to go into a menu. Finally, BrailleBack, Android's app to support reading and writing from a braille display, lacks many of these new capabilities and has greatly fallen behind in terms of its usefulness. Hopefully, attention can be paid to BrailleBack to give it the updates it deserves. This is of vital importance to those who rely on braille as a primary reading method.
All things considered, the fact that Google and Samsung are now combining their efforts to create a single high-quality screen reader for Android is an encouraging sign. As Android 9.1 rolls out to more devices, this will hopefully improve the accessibility experience for more users, regardless of which type of phone they happen to be using, and we look forward to future developments for TalkBack and the Android Accessibility Suite.
This article is made possible in part by generous funding from the James H. and Alice Teubert Charitable Trust, Huntington, West Virginia.
Related Articles:
- An Overview of iOS 14 and iPad OS 14 by Janet Ingber
- An Overview of Apple WatchOS 7 by Janet Ingber
More by this author:
