Discipline: Computer Sciences and Information Management
Subcategory: Computer Science & Information Systems
John Settineri - City College of New York
Vibrotactile sensors are small, infrared devices capable of detecting objects up to two feet away. These devices vibrate when they detect such an object, making them viable for helping visually impaired people (VIP) to navigate. However, testing these devices proves difficult, as they need to approach as wide an audience as possible in order to receive reliable data. The goal of this project was to test the vibrotactile sensors in a virtual environment (VE) created using Unity 3D. The sensors would connect to the virtual environment using Bluetooth technology, and vibrate whenever the virtual player was close to objects. The VE (a house) was designed to cater to visually impaired, as it featured gameplay catering to their enhanced sense of hearing. The player’s goal was to navigate the environment while seeking the target goal: to find a key rustling in a specific room of the house, and escape. To do this, the player needed to rely on the vibrotactile sensors and their own hearing ability, all while interacting with specific objects in the environment to decipher their current location. Video games catered to VIP use a type of gameplay known as binaural audio. Binaural audio games use 3D game engines’ ability to play 3D sound, or sound that becomes louder or softer depending on the player’s position in the 3D environment. However, this type of gameplay was not always effective because players often had trouble knowing their location in relation to the goal. Therefore, a new approach to binaural audio was needed. An interaction button was implemented within the environment to play certain sound effects when pressed. For instance, if the VIP were close to a piano and pressed the interact button, the piano would play a scale to inform the player that a piano was in the room (only one room in the house had a piano). The physics of sound was also taken into account. Since sound varies from surface to surface, a detail noted by VIPs, the VE also had to reflect this. For instance, if the player were walking on a wooden floor-like surface, it would produce a resonant walking sound.
In the future, a multilevel design for the house will be implemented, as it was noted from discussions with VIPs that stairs are a major challenge. The goal of a multilevel environment would be to consider the sensors in other parts of the body besides the upper half. Future improvements that will be considered will come from further interactions and discussions from VIPs and research.
Funder Acknowledgement(s): This work is supported by NSF EFRI-REM under Award # EFRI-1137172, to the City College of New York.
Faculty Advisor: Zhigang Zhu,