A new study in the March 2013 issue of the Journal of Visualized Experiments (JoVE), explores the potential of a virtual gaming environment to help blind individuals improve their navigation skills and develop a cognitive spatial map of unfamiliar buildings and public locations.
The Journal of Visualized Experiments (JoVE) is the first (and still only) PubMed-indexed, peer-reviewed journal devoted to publishing scientific research in a video format. Using an international network of videographers, JoVE films and edits videos of new experimental techniques in the fields of biology, medicine, chemistry, and physics.
The study, entitled Development of an Audio-Based Virtual Gaming Environment to Assist with Navigation Skills in the Blind, was authored by Erin C. Connors, Lindsay A. Yazzolino, Jaime Sánchez, and Lotfi B. Merabet, who are affiliated with the following institutions: Massachusetts Eye and Ear Infirmary, Harvard Medical School; and the Department of Computer Science and Center for Advanced Research in Education, University of Chile.
More about the Research
From the article’s video transcript (used with permission):
Audio-based Environment Simulator (AbES) is virtual environment software designed to improve real-world navigation skills in blind persons. Using only audio-based cues and set within the context of a video game metaphor, users gather relevant spatial information regarding a building’s layout.
This allows the user to develop an accurate spatial cognitive map of a large-scale three-dimensional space that can be manipulated for the purposes of a real indoor navigation task. After game play, participants are then assessed on their ability to navigate within the target physical building represented in the game.
Preliminary results suggest that early blind users were able to acquire relevant information regarding the spatial layout of a previously unfamiliar building as indexed by their performance on a series of navigation tasks. These tasks included pathfinding through the virtual and physical building, as well as a series of drop-off tasks.
We find that the immersive and highly interactive nature of the AbES software appears to greatly engage the blind user to actively explore the virtual environment. Applications of this approach may extend to larger populations of visually impaired individuals.
The article contains a narrated video presentation of the research, a transcript, graphic depictions of the keyboard and test environments, and can be navigated via video article chapters: Title; Preparation and Familiarization with the Audio-based Environment Simulator (AbES); Training and Game Play with AbES; Measuring Virtual Navigation Task Performance; Assess Physical Navigation Task Performance; Assess Physical Drop-off Task Performance; Results; and Conclusion.
The Researchers Explain
From an interview in Science Daily with study author Dr. Lotfi Merabet:
“For [blind persons], finding your way or navigating in a place that is unfamiliar presents a real challenge,” Dr. Merabet explains. “As people with sight, we can capture sensory information through our eyes about our surroundings. For [blind persons], that is a real challenge… blind persons will typically use auditory and tactile cues.”
The technique utilizes computer-generated layouts of public buildings and spatial sensory feedback to synthesize a virtual world that mimics a real world navigation task.
Participants interface with the virtual building by using a keyboard and wearing headphones that play auditory cues that help spatially orient them to the world around them. This interaction helps users generate an accurate mental layout of the mimicked building.
Dr. Merabet and his colleagues are also exploring applications of this technology with other user interfaces, like a Wii Remote or joystick.
“We have developed software called AbES, the Audio Based Environment Simulator that represents the actual physical environment of the Carroll Center for the Blind in Newton, Massachusetts. The participants will use the game metaphor to get a sense of the whole building through open discovery, allowing people to learn room layouts more naturally than if they were just following directions.”
VisionAware will provide updates on this intriguing research as they become available.