About this item:

206 Views | 230 Downloads

Author Notes:

Dr. Tyrone D. Cannon, Department of Psychology, Yale University, P.O. Box 208205, 2 Hillhouse Avenue, New Haven, CT 06520, tyrone.cannon@yale.edu, Phone: (203) 436-1545.

The authors thank the following individuals for assistance with participant scheduling, scan acquisition, data pre-processing and/or programming: Angielette Andaya, Jamie Zinberg Amira Ibrahim, Jared Torre (UCLA); Richard Juelich (BIDMC-Harvard); M. Louis Lauzon, J. Stowkowy and C. Marshall (Calgary); Jason Nunag and Daniel Roman (UCSD); Nicole Popp Santamauro and Hedy Sarofin (Yale).

The authors have no conflicts of interest to report.


Research Funding:

This work was supported by a collaborative U01 award from the National Institute of Mental Health at the National Institutes of Health (MH081902 to TDC; MH081988 to EW; U01 MH081928 to LJS; MH081984 to JA; U01MH066160 to SWW); by the National Center for Research Resources and the National Institute of Biomedical Imaging and Bioengineering of the National Institutes of Health through Grant Number 9P41EB015922 (to AWT); and by the Canada Research Chairs program and the Hopewell Professorship in Brain Imaging (to RF).


  • G-theory
  • Multi-site
  • Reliability
  • Working memory
  • fMRI
  • Adolescent
  • Adult
  • Brain
  • Canada
  • Female
  • Humans
  • Image Processing, Computer-Assisted
  • Longitudinal Studies
  • Magnetic Resonance Imaging
  • Male
  • Memory, Short-Term
  • Multicenter Studies as Topic
  • Oxygen
  • Psychomotor Performance
  • Reaction Time
  • Reproducibility of Results
  • United States
  • Young Adult

Reliability of functional magnetic resonance imaging activation during working memory in a multi-site study: Analysis from the North American Prodrome Longitudinal Study

Show all authors Show less authors


Journal Title:



Volume 97


, Pages 41-52

Type of Work:

Article | Post-print: After Peer Review


Multi-site neuroimaging studies offer an efficient means to study brain functioning in large samples of individuals with rare conditions; however, they present new challenges given that aggregating data across sites introduces additional variability into measures of interest. Assessing the reliability of brain activation across study sites and comparing statistical methods for pooling functional data are critical to ensuring the validity of aggregating data across sites. The current study used two samples of healthy individuals to assess the feasibility and reliability of aggregating multi-site functional magnetic resonance imaging (fMRI) data from a Sternberg-style verbal working memory task. Participants were recruited as part of the North American Prodrome Longitudinal Study (NAPLS), which comprises eight fMRI scanning sites across the United States and Canada. In the first study sample (n= 8), one participant from each home site traveled to each of the sites and was scanned while completing the task on two consecutive days. Reliability was examined using generalizability theory. Results indicated that blood oxygen level-dependent (BOLD) signal was reproducible across sites and was highly reliable, or generalizable, across scanning sites and testing days for core working memory ROIs (generalizability ICCs. = 0.81 for left dorsolateral prefrontal cortex, 0.95 for left superior parietal cortex). In the second study sample (n= 154), two statistical methods for aggregating fMRI data across sites for all healthy individuals recruited as control participants in the NAPLS study were compared. Control participants were scanned on one occasion at the site from which they were recruited. Results from the image-based meta-analysis (IBMA) method and mixed effects model with site covariance method both showed robust activation in expected regions (i.e. dorsolateral prefrontal cortex, anterior cingulate cortex, supplementary motor cortex, superior parietal cortex, inferior temporal cortex, cerebellum, thalamus, basal ganglia). Quantification of the similarity of group maps from these methods confirmed a very high (96%) degree of spatial overlap in results. Thus, brain activation during working memory function was reliable across the NAPLS sites and both the IBMA and mixed effects model with site covariance methods appear to be valid approaches for aggregating data across sites. These findings indicate that multi-site functional neuroimaging can offer a reliable means to increase power and generalizability of results when investigating brain function in rare populations and support the multi-site investigation of working memory function in the NAPLS study, in particular.

Copyright information:

© 2014 Elsevier Inc. All rights reserved.

This is an Open Access work distributed under the terms of the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License (http://creativecommons.org/licenses/by-nc-nd/4.0/).

Creative Commons License

Export to EndNote