Application of the hierarchical bootstrap to multi-level data in neuroscience

• Type of Material: Article
• Authors: Varun Saravanan, Emory University; Gordon Berman, Emory University; Samuel J. Sober, Emory University
• Language: English
• Date: 2020-07-21
• Publisher: NBDT
• Publication Version: Author Accepted Manuscript (After Peer Review)
• Copyright Statement: © 2020 Scholastica
• License: Creative Commons Attribution 4.0 International
• Final Published Version (URL): https://nbdt.scholasticahq.com/article/13927
• Title of Journal or Parent Work: Neurons, Behavior, Data Analysis, and Theory
• Grant/Funding Information: The work for this project was funded by NIH NINDS F31 NS100406, NIH NINDS R01 NS084844, NIH NIBIB R01 EB022872, NIH NIMH R01 MH115831-01, NSF 1456912, Research Corporation for Science Advancement no. 25999 and The Simons Foundation.
• Keywords: hierarchical data structures; neuroscience datasets; bootstrap

Abstract

A common feature in many neuroscience datasets is the presence of hierarchical data structures, most commonly recording the activity of multiple neurons in multiple animals across multiple trials. Accordingly, the measurements constituting the dataset are not independent, even though the traditional statistical analyses often applied in such cases (e.g., Students t-test) treat them as such. The hierarchical bootstrap has been shown to be an effective tool to accurately analyze such data and while it has been used extensively in the statistical literature, its use is not widespread in neuroscience - despite the ubiquity of hierarchical datasets. In this paper, we illustrate the intuitiveness and utility of this approach to analyze hierarchically nested datasets. We use simulated neural data to show that traditional statistical tests can result in a false positive rate of over 45%, even if the Type-I error rate is set at 5%. While summarizing data across non-independent points (or lower levels) can potentially fix this problem, this approach greatly reduces the statistical power of the analysis. The hierarchical bootstrap, when applied sequentially over the levels of the hierarchical structure, keeps the Type-I error rate within the intended bound and retains more statistical power than summarizing methods. We conclude by demonstrating the effectiveness of the method in two real-world examples, first analyzing singing data in male Bengalese finches (Lonchura striata var. domestica) and second quantifying changes in behavior under optogenetic control in flies (Drosophila melanogaster).