Designing COVID-19 mortality predictions to advance clinical outcomes: Evidence from the Department of Veterans Affairs

• Type of Material: Article
• Authors: Christos A Makridis, Department of Veterans Affairs; Tim Strebel, Washington DC VA Medical Center; Vincent Marconi, Emory University; Gil Alterovitz, Department of Veterans Affairs
• Language: English
• Date: 2021-06-09
• Publisher: BMJ
• Publication Version: Final Published Version
• Copyright Statement: © Author(s) (or their employer(s)) 2021. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.
• License: Creative Commons Attribution-NonCommercial 4.0 International
• Final Published Version (URL): http://dx.doi.org/10.1136/bmjhci-2020-100312
• Title of Journal or Parent Work: BMJ Health and Care Informatics
• Volume: 28
• Issue: 1
• Grant/Funding Information: This work was supported by the Department of Veterans Affairs. Apart from the data on demographic characteristics, which are publicly available from the Census Bureau, is restricted to the Department of Veterans Affairs. Christos Makridis contributed to the design, writing, and editing of the paper. Tim Strebel contributed to the analysis. Vince Marconi contributed to the editing of the paper. Gil Alterovitz contributed to the design and editing of the paper.
• Keywords: Artificial Intelligence; Risk Factors; Data Display; medical informatics; information science; United States; Models, Statistical; United States Department of Veterans Affairs; Veterans; Humans; patient care; COVID-19
• Research Categories: Health Sciences, Public Health

Abstract

Using administrative data on all Veterans who enter Department of Veterans Affairs (VA) medical centres throughout the USA, this paper uses artificial intelligence (AI) to predict mortality rates for patients with COVID-19 between March and August 2020. First, using comprehensive data on over 10 000 Veterans' medical history, demographics and lab results, we estimate five AI models. Our XGBoost model performs the best, producing an area under the receive operator characteristics curve (AUROC) and area under the precision-recall curve of 0.87 and 0.41, respectively. We show how focusing on the performance of the AUROC alone can lead to unreliable models. Second, through a unique collaboration with the Washington D.C. VA medical centre, we develop a dashboard that incorporates these risk factors and the contributing sources of risk, which we deploy across local VA medical centres throughout the country. Our results provide a concrete example of how AI recommendations can be made explainable and practical for clinicians and their interactions with patients.