Incorporating minimal user input into deep learning based image segmentation

• Type of Material: Article
• Authors: Maysam Shahedi, University of Texas Dallas; Martin Halicek, University of Texas Dallas; James D. Dormer, University of Texas Dallas; Baowei Fei, Emory University
• Language: English
• Date: 2021-01-01
• Publisher: SPIE Publications
• Publication Version: Author Accepted Manuscript (After Peer Review)
• Copyright Statement: © (2020) Society of Photo-Optical Instrumentation Engineers (SPIE).
• Final Published Version (URL): http://dx.doi.org/10.1117/12.2549716
• Title of Journal or Parent Work: Medical Imaging 2020: Image Processing
• Volume: 11313
• Grant/Funding Information: This research was supported in part by the U.S. National Institutes of Health (NIH) grants (R01CA156775, R01CA204254, R01HL140325, and R21CA231911) and by the Cancer Prevention and Research Institute of Texas (CPRIT) grant RP190588.
• Keywords: deep learning; Science & Technology; prostate; Optics; convolutional neural network (CNN); Physical Sciences; MRI; Imaging Science & Photographic Technology; Engineering; Engineering, Biomedical; Technology; image segmentation
• Research Categories: Health Sciences, Radiology; Engineering, Biomedical; Computer Science

Abstract

Computer-assisted image segmentation techniques could help clinicians to perform the border delineation task faster with lower inter-observer variability. Recently, convolutional neural networks (CNNs) are widely used for automatic image segmentation. In this study, we used a technique to involve observer inputs for supervising CNNs to improve the accuracy of the segmentation performance. We added a set of sparse surface points as an additional input to supervise the CNNs for more accurate image segmentation. We tested our technique by applying minimal interactions to supervise the networks for segmentation of the prostate on magnetic resonance images. We used U-Net and a new network architecture that was based on U-Net (dual-input path [DIP] U-Net), and showed that our supervising technique could significantly increase the segmentation accuracy of both networks as compared to fully automatic segmentation using U-Net. We also showed DIP U-Net outperformed U-Net for supervised image segmentation. We compared our results to the measured inter-expert observer difference in manual segmentation. This comparison suggests that applying about 15 to 20 selected surface points can achieve a performance comparable to manual segmentation.