AI for Visual Feature Detection in Space Applications
Board Location: #130
Discipline: Computer Sciences and Information Management
Session: 3
Ashley Nicole Sutherland - Fayetteville State University
Co-Author(s): Catherine Spooner, Fayetteville State University, Fayetteville, NC; Sambit Bhattacharya, Fayetteville State University, Fayetteville, NC.
There is a strong and ongoing need for the ability to automatically detect, localize and label objects in imagery taken in situ. Currently, in order to determine what pixels belong to a particular class researchers take advantage of crowdsourcing by having a large number of people labeling each pixel with information about what class each object belongs to. For example, all pixels that belong to rocks are given the rock label, while all pixels that include the sky are given a sky label. This process is laborious and prone to mistakes so it would be beneficial to be able to create these label masks automatically. NASA has a strong need for this ability, especially for its rovers.
The data that we used came from the NASA AI4Mars dataset, which are images taken on the Martian surface. The objects that we are interested in labeling are soil, bedrock, sand and big rocks. To solve this problem, we compared 3 methods against the crowd-sourced ground truth. The methods that we used were Facebook’s MaskFormer, GroundingDino with SAM2 and Microsoft’s Florence2 with SAM2. We trained a MaskFormer model with our specific objects of interest. We are in the process of finetuning a GroundingDino with SAM2 detector. Preliminary results indicate that Florence2 is able to pick out rocks with ease, but is not able to distinguish sand from soil. MaskFormer produced the best results so far, while our GroundingDino with SAM2 results indicate that we need to continue to finetune our model.
Future work on this project will be to integrate our segmentation model into a coordinate system so that multiple rovers can map the environment and share the label information among themselves.
Funder Acknowledgement(s): The research and its results outlined in this paper is based upon work supported by NASA under award Nos. 80NSSC21M0312 and 80NSSC23M0054.
Faculty Advisor: Sambit Bhattacharya, sbhattac@uncfsu.edu
Role: As an undergraduate student researcher, I conducted research on various image segmentation models and collaborated with other researchers to finetune these models on the objects of interest for this project.