Reinforcement learning in medical image analysis: Concepts, applications, challenges, and future directions

• Type of Material: Article
• Authors: Mingzhe Hu, Emory University; Jiahan Zhang, Emory University; Luke Matkovic, Emory University; Tian Liu, Emory University; Xiaofeng Yang, Emory University
• Language: English
• Date: 2023-01-10
• Publisher: WILEY
• Publication Version: Final Published Version
• Copyright Statement: © 2023 The Authors. Journal of Applied Clinical Medical Physics published by Wiley Periodicals, LLC on behalf of The American Association of Physicists in Medicine.
• License: Creative Commons Attribution 4.0 International
• Final Published Version (URL): http://dx.doi.org/10.1002/acm2.13898
• Title of Journal or Parent Work: JOURNAL OF APPLIED CLINICAL MEDICAL PHYSICS
• Volume: 24
• Issue: 2
• Start Page: e13898
• End Page: e13898
• Grant/Funding Information: This research is supported in part by the National Institutes of Health under Award Number R01CA215718, R56EB033332 and R01EB032680.
• Keywords: reinforcement learning; Life Sciences & Biomedicine; Radiology, Nuclear Medicine & Medical Imaging; Science & Technology; AGENT
• Research Categories: Computer Science; Health Sciences, Oncology

Abstract

Motivation: Medical image analysis involves a series of tasks used to assist physicians in qualitative and quantitative analyses of lesions or anatomical structures which can significantly improve the accuracy and reliability of medical diagnoses and prognoses. Traditionally, these tedious tasks were finished by experienced physicians or medical physicists and were marred with two major problems, low efficiency and bias. In the past decade, many machine learning methods have been applied to accelerate and automate the image analysis process. Compared to the enormous deployments of supervised and unsupervised learning models, attempts to use reinforcement learning in medical image analysis are still scarce. We hope that this review article could serve as the stepping stone for related research in the future. Significance: We found that although reinforcement learning has gradually gained momentum in recent years, many researchers in the medical analysis field still find it hard to understand and deploy in clinical settings. One possible cause is a lack of well-organized review articles intended for readers without professional computer science backgrounds. Rather than to provide a comprehensive list of all reinforcement learning models applied in medical image analysis, the aim of this review is to help the readers formulate and solve their medical image analysis research through the lens of reinforcement learning. Approach & Results: We selected published articles from Google Scholar and PubMed. Considering the scarcity of related articles, we also included some outstanding newest preprints. The papers were carefully reviewed and categorized according to the type of image analysis task. In this article, we first reviewed the basic concepts and popular models of reinforcement learning. Then, we explored the applications of reinforcement learning models in medical image analysis. Finally, we concluded the article by discussing the reviewed reinforcement learning approaches’ limitations and possible future improvements.