Last year, Microsoft released its Seeing AI OCR and recognition app on the iOS platform, which received an overwhelmingly positive reception from the blind and visually impaired community. Recently, Microsoft has released a second app on the iOS platform aimed at aiding people with vision loss: a unique wayfinding app called Soundscape. The Soundscape app is novel in the wayfinding space as it provides information about points of interest and the streets around you using augmented reality by way of 3D sound. Since its release, the app has seen several updates and the development team is always seeking feedback from the community so that they can make the app as helpful as possible.
In this article, I will detail Soundscape's features and operation and also provide my thoughts and findings when using Soundscape in real world situations.
Getting Started and App Layout
After downloading Soundscape from the app store and launching it for the first time, you will be presented with a welcome screen and asked to grant the app the permissions it will need to operate effectively. First you will be asked to choose a voice that will be used to identify landmarks and points of interest. The default is a female UK voice but two male voices (US and UK) and a female US voice are also available to download. After making your selection, you will begin navigating through a series of screens asking you to grant permissions including location access and motion/fitness activity. Soundscape will describe why it needs these permissions and, once you grant permission, you will be presented with the standard iOS dialog for allowing the app access to this information. After permissions have been granted, you will be shown a video demonstrating a Soundscape audio beacon as well as a point of interest callout. After the video several of Soundscape's features are described; you can navigate through these by either changing the page in the Picker or by tapping the "Next Page" button. After the last page, you will be asked to agree to the Soundscape terms and conditions. Note that the terms and conditions are not displayed on this page; use the link located in the text before the check box to view them.
Once you have agreed to the Terms and Conditions, you are taken to the Soundscape Home screen. The Home screen has a simple layout and contains almost all of the controls that you would commonly use.
In the top left of the Home screen you will find a Menu button where you can manage various settings and access help files and tutorials. Across from this button is a Callouts button, which will allow you to turn the audio announcements provided by Soundscape on or off. Below these two buttons are two large buttons that cover the width of the screen. First is the Set a Beacon button, which will allow you to set an audio beacon on an address or point of interest, followed by an Expand Callout History button below. This button will produce a horizontally scrolling list of recent announcements from the text-to-speech (TTS) voice so that you can review them. You can close the history by activating the button again. At the bottom of the screen is a row of three buttons. From left to right these are My Location, Around Me, and Ahead of Me. These three buttons will cause the TTS voice to announce the requested information about your surroundings. Note that often controls on the home screen will have options through the "Actions" item in the VoiceOver Rotor. For example, a point of interest displayed in the Callout History will have options to set a beacon on the location, repeat it, or gain more information about the point.
3D Sound through Soundscape
The main feature that makes Soundscape unique is how it presents information. All information provided by the TTS voice, as well as the audio beacons, are presented in 3D audio. This means that if you are passing a point of interest on your left, it will sound from your left earphone. The way the sound is presented, it sounds as if it originates from the outside environment to a greater or lesser degree depending on the earphones you are using. To me, it sounds as if the text-to-speech is coming from somewhat above me, giving callouts the sound of audible signs. Beacons appear as a steady drumbeat in the direction of the location of the beacon. If you are facing the beacon's direction so that it is in front of you, a steady beep will play along with the beat. Again, the way that the beacon is presented sounds as if it is coming from the environment and blends somewhat with the other outside sounds. To hear a demonstration of both callouts and beacons, see this YouTube Video.
Callouts and Gaining Information
There are two primary ways of gaining information from Soundscape, automatic callouts and requesting information directly using the buttons at the bottom of the Home screen. As you travel, Soundscape will automatically call out certain information. By default, Soundscape announces points of interest as you pass them, upcoming intersections, transportation information, and the distance to a beacon, if you have one set. It is possible to manage what is spoken by selecting the Menu button and then the Manage Callouts item. In addition to the previously mentioned callouts, you can choose to have indoor Bluetooth beacons (where they are present) and current location information spoken.
As you walk, the callouts that you have set will automatically be announced. Points of interest are announced as you pass them while intersection information is usually read somewhat prior to the intersection. As mentioned previously, the TTS voice speaks from the position of the point of interest. For example, if you are standing in front of a row of buildings, those to the extreme left and right would be spoken from the very far left and right, but as the TTS announced points of interest that were closer to directly in front of you, they would be announced closer to the center of the stereo field. When approaching an intersection, the TTS will notify you which streets are left, right, and ahead. Again, streets to the left and right will be spoken from their respective points in the stereo field, while streets that continue ahead will be announced from the center of the field. Announcements of position information, such as "Facing southeast along Main Street," are spoken from the center of the audio field. You can turn and the position of the callout will remain in the direction of the point of interest. As an example, if you are walking and hear a point of interest on your left and turn to face it while the voice is speaking, the speech will move through the stereo field to be in front of you when you are facing it.
To request information manually, you can use the three buttons across the bottom of the Home screen. The My Location button will provide your current compass direction as well as the street on which you are walking. The Around Me button will alert you to four nearby points of interest, one each behind, to your right, left, and in front of you. The Ahead of Me button will list five points of interest that are in front of you. Note that distance does not seem to be a factor in what is reported. For example, I was in an area with few points of interest so was provided information on locations up to a half a mile away. Another useful aspect of the Ahead of Me information is that if you are unsure which points of interest reported by the Around Me button are ahead or behind you, you can use the Ahead of Me button to confirm. Note that you can gain more information about a specific callout or interact with it from the Callout History section on the home screen.
Beacons
To add a beacon, select the Add a Beacon icon from the home screen. When you add a beacon for the first time, you will be presented with a tutorial that will walk you through the process. Interestingly, the tutorial uses a beacon in your area as an example and references it in the instructions instead of using pre-scripted examples. If you choose to skip the tutorial, it will be available in the Help and Tutorials section of the menu.
When you place a beacon on a location or address, you will hear a steady beat from the direction of the beacon. Be aware that the beacon changes position in the stereo field depending on your orientation in relation to it, but does not get louder or softer depending on your proximity. As you travel with a beacon active, the TTS will regularly provide updates on your distance to the beacon's location. Once you reach the beacon the TTS will announce it without any distance information. The beacon should stop sounding at this point, but in my tests, I found that I had to stop walking soon after the beacon was announced or else the beacon would not automatically end. If I kept walking, such as in the case where a beacon was not directly on the entrance of a large building, the beacon would continue to sound.
Soundscape in the Real World
Now that we've explored how Soundscape functions, let's look at how effective it is in real world situations. I've found that in most situations Soundscape functions well but is highly reliant on the quality of the maps in your area. Soundscape uses Open Street Map (OSM) for its map data, which has variable accuracy depending on your location. In large cities, the data seems to be full featured and up to date while in more rural areas there can be many points of interest missing or very old data. When using Soundscape in a large city I found that I was given a great deal of information and as long as the maps were up to date, Soundscape could identify points of interest with a high degree of accuracy. In my home city, there are few labeled points of interest so it has been less useful for discovering new points of interest, but I have found it useful for identifying streets and intersections. Fortunately, Soundscape allows you to set a beacon on an address so it is possible to use a beacon to navigate to a location even if it is not labeled. A feature called Markers is planned to be included in the app as of the next release which would allow you to mark Points of interest, addresses, and your current location. This would allow you to fill in gaps in your local map data (only for your own device) and mark your own personal landmarks such as building entrances, park benches, or patches of grass for a guide dog. Keep an eye on AccessWorld News, as we will be announcing the availability of the feature there once it has been released to the app store.
I have found Soundscape to be quite accurate in its positioning of callouts and beacons in the stereo field. In some situations I have noticed beacons switching orientation even though I am not changing my direction, but in almost all cases, the beacon settled in the correct orientation as I continued walking. This generally occurred after making a turn so may have been due to compass calibration issues. In areas with many up-to-date points of interest, I have been able to use Soundscape beacons to locate my destination. In my home city where points of interest are more scarce, I have still found address-based beacons to be very accurate. I've found Soundscape to be most useful paired with a turn-by-turn navigation app such as Google Maps. The turn-by-turn directions meshed well with the callouts provided by Soundscape. This would be particularly helpful if you are navigating in an unfamiliar area where gaining intersection information is most important. I found that Soundscape could be a significant drain on my battery. For my testing I was using an iPhone SE, so newer generations or larger devices may fare better, but I found it necessary to close Soundscape from the App Switcher to be sure it wouldn't drain my battery after use.
The Bottom Line
Soundscape adds a unique and novel dimension to travel and presents a fairly seamless method for understanding your surroundings, particularly in areas with solid maps. Like any GPS app, there are some accuracy issues, but in my testing, these errors were rare and correct themselves relatively quickly. For anyone with an iOS device, I highly recommend giving Soundscape a try; the app is free and even if you already have a solid navigation strategy, the unique nature of Soundscape's presentation is enjoyable to experience and you may find it a useful addition to your toolkit.
Product Information
Product: Microsoft Soundscape
Platform: iOS
Price: Free Developer: Microsoft
Developer Comments
We recognize Soundscape is a new technology. As such, it does take a bit of getting used to. Its intention is to increace a person's awareness of where they are and what's around them in a pretty novel way. Part of our design ethos is to help people get into a position where they can exercise greater choice about how they get from one place to another, so they can start to connect with more of the world around them.
Because Soundscape is new and novel, we are very keen to receive feedback and learn what is working well and what isn't, so that we can work with our insiders and partners to figure out ways to improve the overall experience.
This article is made possible in part by generous funding from the James H. and Alice Teubert Charitable Trust, Huntington, West Virginia.
Related articles:
- Feel This: The Current State of Braille with the Windows Narrator Screen Reader by Scott Davert
- Microsoft Seeing AI: A Quick First Look at this Groundbreaking iOS App by Bill Holton
More by this author:
