Reliability of functional magnetic resonance imaging activation during working memory in a multi-site study: Analysis from the North American Prodrome Longitudinal Study

• Type of Material: Article
• Authors: Jennifer K. Forsyth, University of California, Los Angeles; Sarah C. McEwen, University of California, Los Angeles; Dylan G. Gee, University of California, Los Angeles; Carrie E. Bearden, University of California, Los Angeles; Jean Addington, University of Calgary; Brad Goodyear, University of Calgary; Kritin S. Cadenhead, University of California, San Diego; Heline Mirzakhanian, University of California, San Diego; Barbara A. Cornblatt, Zucker Hillside Hospital; Doreen M. Olvet, Zucker Hillside Hospital; Daniel H. Mathalon, University of California, San Francisco; Thomas H. McGlashan, Yale University; Diana O. Perkins, University of North Carolina at Chapel Hill; Aysenil Belger, University of North Carolina at Chapel Hill; Larry J. Seidman, Beth Israel Deaconess Medical Center; Heidi W. Thermenos, Beth Israel Deaconess Medical Center; Ming T. Tsuang, University of California, San Diego; Theo G. M. van Erp, University of California, Irvine; Elaine Walker, Emory University; Stephan Hamann, Emory University; Scott W. Woods, Yale University; Maolin Qiu, Yale University; Tyrone D. Cannon, Yale University
• Language: English
• Date: 2014-08-15
• Publisher: Elsevier
• Publication Version: Author Accepted Manuscript (After Peer Review)
• Copyright Statement: © 2014 Elsevier Inc. All rights reserved.
• License: Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International
• Final Published Version (URL): http://dx.doi.org/10.1016/j.neuroimage.2014.04.027
• Title of Journal or Parent Work: NeuroImage
• ISSN: 1053-8119
• Volume: 97
• Start Page: 41
• End Page: 52
• Grant/Funding Information: This work was supported by a collaborative U01 award from the National Institute of Mental Health at the National Institutes of Health (MH081902 to TDC; MH081988 to EW; U01 MH081928 to LJS; MH081984 to JA; U01MH066160 to SWW); by the National Center for Research Resources and the National Institute of Biomedical Imaging and Bioengineering of the National Institutes of Health through Grant Number 9P41EB015922 (to AWT); and by the Canada Research Chairs program and the Hopewell Professorship in Brain Imaging (to RF).
• Keywords: Brain; Magnetic Resonance Imaging; Young Adult; Psychomotor Performance; Oxygen; Male; Canada; Multicenter Studies as Topic; Longitudinal Studies; Reproducibility of Results; Multi-site; G-theory; Reliability; United States; Female; Humans; Image Processing, Computer-Assisted; Memory, Short-Term; Working memory; fMRI; Reaction Time; Adult; Adolescent
• Research Categories: Psychology, Clinical; Biology, Neuroscience; Health Sciences, Radiology

Abstract

Multi-site neuroimaging studies offer an efficient means to study brain functioning in large samples of individuals with rare conditions; however, they present new challenges given that aggregating data across sites introduces additional variability into measures of interest. Assessing the reliability of brain activation across study sites and comparing statistical methods for pooling functional data are critical to ensuring the validity of aggregating data across sites. The current study used two samples of healthy individuals to assess the feasibility and reliability of aggregating multi-site functional magnetic resonance imaging (fMRI) data from a Sternberg-style verbal working memory task. Participants were recruited as part of the North American Prodrome Longitudinal Study (NAPLS), which comprises eight fMRI scanning sites across the United States and Canada. In the first study sample (n= 8), one participant from each home site traveled to each of the sites and was scanned while completing the task on two consecutive days. Reliability was examined using generalizability theory. Results indicated that blood oxygen level-dependent (BOLD) signal was reproducible across sites and was highly reliable, or generalizable, across scanning sites and testing days for core working memory ROIs (generalizability ICCs. = 0.81 for left dorsolateral prefrontal cortex, 0.95 for left superior parietal cortex). In the second study sample (n= 154), two statistical methods for aggregating fMRI data across sites for all healthy individuals recruited as control participants in the NAPLS study were compared. Control participants were scanned on one occasion at the site from which they were recruited. Results from the image-based meta-analysis (IBMA) method and mixed effects model with site covariance method both showed robust activation in expected regions (i.e. dorsolateral prefrontal cortex, anterior cingulate cortex, supplementary motor cortex, superior parietal cortex, inferior temporal cortex, cerebellum, thalamus, basal ganglia). Quantification of the similarity of group maps from these methods confirmed a very high (96%) degree of spatial overlap in results. Thus, brain activation during working memory function was reliable across the NAPLS sites and both the IBMA and mixed effects model with site covariance methods appear to be valid approaches for aggregating data across sites. These findings indicate that multi-site functional neuroimaging can offer a reliable means to increase power and generalizability of results when investigating brain function in rare populations and support the multi-site investigation of working memory function in the NAPLS study, in particular.