A Prospective Observational Study to Investigate Performance of a Chest X-ray Artificial Intelligence Diagnostic Support Tool Across 12 U.S. Hospitals.

• Type of Material: Article
• Authors: Ju Sun, University of Minnesota; Le Peng, University of Minnesota; Taihui Li, University of Minnesota; Dyah Adila, University of Minnesota; Zach Zaiman, Emory University; Genevieve B. Melton, University of Minnesota; Nicholas Ingraham, University of Minnesota; Eric Murray, M Health Fairview Informatics; Daniel Boley, University of Minnesota; Sean Switzer, University of Minnesota; John L. Burns, Indiana University; Kun Huang, Indiana University; Tadashi Allen, University of Minnesota; Scott D. Steenburg, Indiana University; Judy Gichoya, Emory University; Erich Kummerfeld, University of Minnesota; Christopher Tignanelli, University of Minnesota
• Language: English
• Date: 2021-06-03
• Publisher: NIH
• Publication Version: Preprint (Prior to Peer Review)
• Copyright Statement: The copyright holder for this preprint is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.
• License: Creative Commons Attribution 4.0 International
• Final Published Version (URL): http://dx.doi.org/10.1101/2021.06.04.21258316
• Title of Journal or Parent Work: ArXiv
• Grant/Funding Information: NIH NHLBI T32HL07741 (NEI); This research was supported by the University of Minnesota Office of the Vice President of Research (OVPR) COVID-19 Rapid Response Grants (JS, EK, CJT); This research was supported by the Agency for Healthcare Research and Quality (AHRQ) and Patient-Centered Outcomes Research Institute (PCORI), grant K12HS026379 (CJT) and the National Institutes of Health’s National Center for Advancing Translational Sciences, grants KL2TR002492 (CJT) and UL1TR002494 (EK).; NIH NIBIB 75N92020D00018/75N92020F00001 (JWG)
• Keywords: CXR; Open-I 2013 Chest X-ray Collection; receiver operating characteristic (ROC); F1 score; COVID-19; Precision-Recall curves

Abstract

IMPORTANCE: An artificial intelligence (AI)-based model to predict COVID-19 likelihood from chest x-ray (CXR) findings can serve as an important adjunct to accelerate immediate clinical decision making and improve clinical decision making. Despite significant efforts, many limitations and biases exist in previously developed AI diagnostic models for COVID-19. Utilizing a large set of local and international CXR images, we developed an AI model with high performance on temporal and external validation. CONCLUSIONS AND RELEVANCE: AI-based diagnostic tools may serve as an adjunct, but not replacement, for clinical decision support of COVID-19 diagnosis, which largely hinges on exposure history, signs, and symptoms. While AI-based tools have not yet reached full diagnostic potential in COVID-19, they may still offer valuable information to clinicians taken into consideration along with clinical signs and symptoms.