Multimodal Assessment of Schizophrenia and Depression Utilizing Video, Acoustic, Locomotor, Electroencephalographic, and Heart Rate Technology: Protocol for an Observational Study

• Type of Material: Article
• Authors: Robert Cotes, Emory University; Mina Boazak, Animo Sano Psychiatry; Emily Griner, Emory University; Zifan Jiang, Emory University; Bona Kim, Emory University; Whitney Bremer, Emory University; Salman Seyedi, Emory University; Ali Bahrami Rad, Emory University; Gari Clifford, Emory University
• Language: English
• Date: 2022-07-01
• Publisher: JMIR PUblications
• Publication Version: Final Published Version
• Copyright Statement: ©Robert O Cotes, Mina Boazak, Emily Griner, Zifan Jiang, Bona Kim, Whitney Bremer, Salman Seyedi, Ali Bahrami Rad, Gari D Clifford. Originally published in JMIR Research Protocols (https://www.researchprotocols.org), 13.07.2022.
• License: Creative Commons Attribution 4.0 International
• Final Published Version (URL): http://dx.doi.org/10.2196/36417
• Title of Journal or Parent Work: JMIR Research Protocols
• Volume: 11
• Issue: 7
• Start Page: e36417
• End Page: e36417
• Keywords: biomarker; computer vision; digital biomarker; depression; schizophrenia; acoustic; multimodal; technology; machine learning; heart rate
• Research Categories: Health Sciences, Mental Health; Engineering, Biomedical

Abstract

Background: Current standards of psychiatric assessment and diagnostic evaluation rely primarily on the clinical subjective interpretation of a patient's outward manifestations of their internal state. While psychometric tools can help to evaluate these behaviors more systematically, the tools still rely on the clinician's interpretation of what are frequently nuanced speech and behavior patterns. With advances in computing power, increased availability of clinical data, and improving resolution of recording and sensor hardware (including acoustic, video, accelerometer, infrared, and other modalities), researchers have begun to demonstrate the feasibility of cutting-edge technologies in aiding the assessment of psychiatric disorders. Objective: We present a research protocol that utilizes facial expression, eye gaze, voice and speech, locomotor, heart rate, and electroencephalography monitoring to assess schizophrenia symptoms and to distinguish patients with schizophrenia from those with other psychiatric disorders and control subjects. Methods: We plan to recruit three outpatient groups: (1) 50 patients with schizophrenia, (2) 50 patients with unipolar major depressive disorder, and (3) 50 individuals with no psychiatric history. Using an internally developed semistructured interview, psychometrically validated clinical outcome measures, and a multimodal sensing system utilizing video, acoustic, actigraphic, heart rate, and electroencephalographic sensors, we aim to evaluate the system's capacity in classifying subjects (schizophrenia, depression, or control), to evaluate the system's sensitivity to within-group symptom severity, and to determine if such a system can further classify variations in disorder subtypes. Results: Data collection began in July 2020 and is expected to continue through December 2022. Conclusions: If successful, this study will help advance current progress in developing state-of-the-art technology to aid clinical psychiatric assessment and treatment. If our findings suggest that these technologies are capable of resolving diagnoses and symptoms to the level of current psychometric testing and clinician judgment, we would be among the first to develop a system that can eventually be used by clinicians to more objectively diagnose and assess schizophrenia and depression with the possibility of less risk of bias. Such a tool has the potential to improve accessibility to care; to aid clinicians in objectively evaluating diagnoses, severity of symptoms, and treatment efficacy through time; and to reduce treatment-related morbidity.