OSCARS: An Outlier-Sensitive Content-Based Radiography Retrieval System

• Type of Material: Article
• Authors: Xiaoyuan Guo, Emory University; Jiali Duan, University of Southern California; Saptarshi Purkayastha, Indiana University; Hari Trivedi, Emory University; Judy Gichoya, Emory University; Imon Banerjee, Arizona State University
• Language: English
• Date: 2022-06-27
• Publisher: NIH
• Publication Version: Preprint (Prior to Peer Review)
• Copyright Statement: © 2022 ACM.
• Final Published Version (URL): http://dx.doi.org/10.1145/3512527.3531425
• Title of Journal or Parent Work: ICMR 2022 - Proceedings of the 2022 International Conference on Multimedia Retrieval
• Start Page: 11
• End Page: 18

Abstract

Improving the retrieval relevance on noisy datasets is an emerging need for the curation of a large-scale clean dataset in the medical domain. While existing methods can be applied for class-wise retrieval (aka. inter-class), they cannot distinguish the granularity of likeness within the same class (aka. intra-class). The problem is exacerbated on medical external datasets, where noisy samples of the same class are treated equally during training. Our goal is to identify both intra/inter-class similarities for fine-grained retrieval. To achieve this, we propose an Outlier-Sensitive Content-based rAdiologhy Retrieval System (OSCARS), consisting of two steps. First, we train an outlier detector on a clean internal dataset in an unsupervised manner. Then we use the trained detector to generate the anomaly scores on the external dataset, whose distribution will be used to bin intra-class variations. Second, we propose a quadruplet (a, p, nintra, ninter) sampling strategy, where intra-class negatives nintra are sampled from bins of the same class other than the bin anchor a belongs to, while n_inter are randomly sampled from inter-classes. We suggest a weighted metric learning objective to balance the intra and inter-class feature learning. We experimented on two representative public radiography datasets. Experiments show the effectiveness of our approach. The training and evaluation code can be found in https://github.com/XiaoyuanGuo/oscars.