A Deep Deterministic Policy Gradient Approach to Medication Dosing and Surveillance in the ICU

• Type of Material: Article
• Authors: Rongmei Lin, Emory University; Matthew D. Stanley, Emory University; Mohammad M. Ghassemi, Massachusetts Institute of Technology; Shamim Nemati, Emory University
• Language: English
• Date: 2018-10-26
• Publisher: IEEE
• Publication Version: Author Accepted Manuscript (After Peer Review)
• Copyright Statement: © 2018 IEEE.
• Final Published Version (URL): http://dx.doi.org/10.1109/EMBC.2018.8513203
• Title of Journal or Parent Work: Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS
• Volume: 2018-July
• Start Page: 4927
• End Page: 4931
• Grant/Funding Information: NIH early career development award in Biomedical Big Data science (1K01ES025445-01A1); NIH training Grants: T32EB001680, T90DA22759, T32HL007901.
• Keywords: Anticoagulants; Critical Care; Reinforcement, Psychology; Humans; Heparin; Intensive Care Units
• Research Categories: Health Sciences, Medicine and Surgery; Health Sciences, Pharmacy

Abstract

Medication dosing in a critical care environment is a complex task that involves close monitoring of relevant physiologic and laboratory biomarkers and corresponding sequential adjustment of the prescribed dose. Misdosing of medications with narrow therapeutic windows (such as intravenous [IV] heparin) can result in preventable adverse events, decrease quality of care and increase cost. Therefore, a robust recommendation system can help clinicians by providing individualized dosing suggestions or corrections to existing protocols. We present a clinician-in-the-loop framework for adjusting IV heparin dose using deep reinforcement learning (RL). Our main objectives were to learn a new IV heparin dosing policy based on the multi-dimensional features of patients, and evaluate the effectiveness of the learned policy in the presence of other confounding factors that may contribute to heparin-related side effects. The data used in the experiments included 2598 intensive care patients from the publicly available MIMIC database and 2310 patients from the Emory University clinical data warehouse. Experimental results suggested that the distance from RL policy had a statistically significant association with anticoagulant complications (p< 0.05), after adjusting for the effects of confounding factors.