Discipline: Technology and Engineering
Subcategory: Computer Science & Information Systems
Kenichi Yamamoto - City College of New York
Co-Author(s): Feng Hu, The City College and CUNY Graduate Center, New York City, NY Zhigang Zhu, The City College of New York, New York City, NY
An accurate, real-time, and robust indoor localization system with a natural interface is important, as it allows visually impaired people to live a convenient social life. This project aims to build a Google Glass application which can in real-time determine the device’s camera position and orientation within indoor environments, such as in campus buildings or rooms at home. Pre-captured 2D images are used to reconstruct the indoor environment as a 3D model using VSFM (Visual Structure from Motion). The keypoint descriptors extracted from a Google Glass image of the environment are then matched to the precaptured image’s descriptors using FLANN (Fast Library for Approximate Nearest Neighbors). After finding each descriptor’s related 3D point within the model, PnP (Perspective-n-Point) and RANSAC (Random Sample Consensus) are then used to estimate the camera’s pose, which is the orientation and location of the camera. Currently, the pose result obtained from this method is within several centimeters from the ground truth position. It takes a total time of around ten seconds from the start of the program’s execution to the pose result, including the initial time taken to load all the 3D model data. After implementing the code onto the Google Glass, the model and pose can then be used in conjunction with one another to detect nearby points of interest within a 2D projected map of the environment. This result can then be used to give the user simple navigation instructions via an integrated speaker on the Google Glass.
Funder Acknowledgement(s): This work is supported by NSF EFRI-REM under Award # EFRI-1137172, to the City College of New York.
Faculty Advisor: Zhigang Zhu,