Neural basis of the sound-symbolic crossmodal correspondence between auditory pseudowords and visual shapes

• Authors: Kelly McCormick, Emory University; Simon Lacey, Emory University; Randall Stilla, Emory University; Lynne Nygaard, Emory University; Krishnankutty Sathian, Emory University
• Journal Title: Multisensory Research
• Volume: Volume 35, Number 1
• Publisher: Brill | 2021-01-01, Pages 29-78
• Type of Work: Article | Post-print: After Peer Review
• Publisher DOI: http://doi.org/10.1163/22134808-bja10060
• Subjects: Health Sciences, Medicine and Surgery; Health Sciences, Oncology
• Keywords: Auditory Perception; Brain Mapping; Humans; Language; Magnetic Resonance Imaging; Sound; Symbolism

Abstract

Sound symbolism refers to the association between the sounds of words and their meanings, often studied using the crossmodal correspondence between auditory pseudowords, e.g., 'takete' or 'maluma', and pointed or rounded visual shapes, respectively. In a functional magnetic resonance imaging study, participants were presented with pseudoword-shape pairs that were soundsymbolically congruent or incongruent. We found no significant congruency effects in the blood oxygenation level-dependent (BOLD) signal when participants were attending to visual shapes. During attention to auditory pseudowords, however, we observed greater BOLD activity for incongruent compared to congruent audiovisual pairs bilaterally in the intraparietal sulcus and supramarginal gyrus, and in the left middle frontal gyrus. We compared this activity to independent functional contrasts designed to test competing explanations of sound symbolism, but found no evidence for mediation via language, and only limited evidence for accounts based on multisensory integration and a general magnitude system. Instead, we suggest that the observed incongruency effects are likely to reflect phonological processing and/or multisensory attention. These findings advance our understanding of sound-to-meaning mapping in the brain.