Discipline: Computer Sciences & Information Management
Subcategory: STEM Science and Mathematics Education
Mahmudur Rahman - Morgan State University, Baltimore, MD
Co-Author(s): Ibukunoluwa Oladunjoye, Morgan State University, Baltimore, MD
Authors of biomedical journal articles frequently use graphical and photographic images to illustrate the medical concepts or to highlight regions-of-interests (ROIs) by using annotation markers (pointers) such as arrows, letters or symbols overlaid on figures. Localizing and recognizing the markers can assist in extracting relevant image content at regions within the image that are likely to be highly relevant to the discussion in the article text. Image regions can then be annotated using biomedical concepts from extracted snippets of text that are identified using existing textual ontologies, such as UMLS. Whereas ontological resources exist for relating the meaning of textual entities, no such resource exists for relating the appearance of visual entities due to the lack of ground truth data set in medical domain. To overcome this limitation, this work focuses on developing an online crowdsourcing based annotation tool with the goal of large scale modality classification and concept. We designed and developed an Amazon Mechanical Turk (MTurk) based annotation tool with an engaging interface, clear task description and interface usage instructions, as well as proper verification and quality control. In this tool, users (aka workers) are able to perform image categorization task based on displayed images with associated caption in the interface as HIT (Human Intelligence Task ¬) and annotate image concepts (ROI) by manually drawing polygons based on observing the location of pointers and their description in associated image captions. In addition, by automatically mapping the captions to UMLS concepts, additional keywords/concept categories are provided to the interface for further selection and refinement by users. Our goal is to collect annotation information through MTurk Requester Interface, analyze the data for further verification and cleaning purposes and generate training sets for modality classification and concept detection tasks. It is expected that the online tool will able to generate data which would offer building blocks for the development of advanced information retrieval systems aided by a visual ontology. This poster will consist of two panels: The first panel will demonstrate the work flow of the MTurk process using Requester Interface and second panel will display our annotation interface with example HITs, submitted tasks, and result generation for further analysis to generate the ground truth.
Funder Acknowledgement(s): This study is supported by a grant from NSF HBCU-UP (Research Initiation Award) awarded to Md Mahmudur Rahman PhD, Assistant Professor, Computer Science Department, Morgan State University, Baltimore, Maryland.
Faculty Advisor: None Listed,