Synesthesia is a phenomenon in which experiences in one sensory or cognitive domain are associated with automatic, involuntary experiences in a second domain. The present study investigated the relationship between the consistency and strength of these associations in grapheme-color synesthesia, in which a specific color is experienced when seeing a particular letter or number. Firstly, synesthetic participants completed the online Synesthesia Battery (SB) which measures the consistency with which individuals choose the same color for the same grapheme and returns a standardized score which distinguishes genuine synesthetes from non-synesthetes. Secondly, synesthetes and age/gender-matched non-synesthetic control participants completed an Implicit Association Test (IAT) which measures the strength of associations. In the IAT, two response keys were paired with the synesthetes’ two most consistent SB associations in either congruent (each key is associated with a grapheme and its correct synesthetic color, e.g., A + red, B + green) or incongruent (i.e., A + green, B + red) conditions. However, on each trial, only a single grapheme or color is presented and participants make speeded responses. We expected that synesthetes would respond more quickly and accurately when their grapheme/color associations were paired congruently (e.g., A/red, B/blue) as opposed to incongruently (i.e., A/blue, B/red). In contrast, non-synesthetic controls should show no significant difference between congruent and incongruent trials because they do not have pre-existing associations between graphemes and colors. To the extent that strong associations should also be consistent, we also expected a positive correlation between SB scores and congruency magnitudes in the synesthetes.

It is often assumed that a fundamental property of language is the arbitrariness of the relationship between sound and meaning. Sound symbolism, which refers to non-arbitrary mapping between the sound of a word and its meaning, contradicts this assumption. Sensitivity to sound symbolism has been studied through crossmodal correspondences (CCs) between auditory pseudowords (e.g. ‘loh-moh’) and visual shapes (e.g. a blob). We used representational similarity analysis to examine the relationships between physical stimulus parameters and perceptual ratings that varied on dimensions of roundedness and pointedness, for a range of auditory pseudowords and visual shapes. We found that perceptual ratings of these stimuli relate to certain physical features of both the visual and auditory domains. Representational dissimilarity matrices (RDMs) of parameters that capture the spatial profile of the visual shapes, such as the simple matching coefficient and Jaccard distance, were significantly correlated with those of the visual ratings. RDMs of certain acoustic parameters of the pseudowords, such as the temporal fast Fourier transform (FFT) and spectral tilt, that reflect spectral composition, as well as shimmer and speech envelope that reflect aspects of amplitude variation over time, were significantly correlated with those of the auditory perceptual ratings. RDMs of the temporal FFT (acoustic) and the simple matching coefficient (visual) were significantly correlated. These findings suggest that sound-symbolic CCs are related to basic properties of auditory and visual stimuli, and thus provide insights into the fundamental nature of sound symbolism and how this might evoke specific impressions of physical meaning in natural language.

The present study investigated the degree to which perceptual adaptation to foreign-accented speech is specific to the regularities in pronunciation associated with a particular accent. Across experiments, the conditions under which generalization of learning did or did not occur were evaluated. In Experiment 1, listeners trained on word-length utterances in Korean-accented English and tested with words produced by the same or different set of Korean-accented speakers. Listeners performed better than untrained controls when tested with novel words from the same or different speakers. In Experiment 2, listeners were trained with Spanish-, Korean-, or mixed-accented speech and transcribed novel words produced by unfamiliar Korean- or Spanish-accented speakers at test. The findings revealed relative specificity of learning. Listeners trained and tested on the same variety of accented speech showed better transcription at test than those trained with a different accent or untrained controls. Performance after mixed-accent training was intermediate. Patterns of errors and analysis of acoustic properties for accented vowels suggested perceptual improvement for regularities arising from each accent, with learning dependent on the relative similarity of linguistic form within and across accents. © 2019 Acoustical Society of America.

Listeners infer which object in a visual scene a speaker refers to from the systematic variation of the speaker’s tone of voice (ToV). We examined whether ToV also guides word learning. During exposure, participants heard novel adjectives (e.g., “daxen”) spoken with a ToV representing hot, cold, strong, weak, big, or small while viewing picture pairs representing the meaning of the adjective and its antonym (e.g., elephant-ant for big-small). Eye fixations were recorded to monitor referent detection and learning. During test, participants heard the adjectives spoken with a neutral ToV, while selecting referents from familiar and unfamiliar picture pairs. Participants were able to learn the adjectives’ meanings, and, even in the absence of informative ToV, generalise them to new referents. A second experiment addressed whether ToV provides sufficient information to infer the adjectival meaning or needs to operate within a referential context providing information about the relevant semantic dimension. Participants who saw printed versions of the novel words during exposure performed at chance during test. ToV, in conjunction with the referential context, thus serves as a cue to word meaning. ToV establishes relations between labels and referents for listeners to exploit in word learning.

Sound symbolism refers to the association between the sounds of words and their meanings, often studied using the crossmodal correspondence between auditory pseudowords, e.g., 'takete' or 'maluma', and pointed or rounded visual shapes, respectively. In a functional magnetic resonance imaging study, participants were presented with pseudoword-shape pairs that were soundsymbolically congruent or incongruent. We found no significant congruency effects in the blood oxygenation level-dependent (BOLD) signal when participants were attending to visual shapes. During attention to auditory pseudowords, however, we observed greater BOLD activity for incongruent compared to congruent audiovisual pairs bilaterally in the intraparietal sulcus and supramarginal gyrus, and in the left middle frontal gyrus. We compared this activity to independent functional contrasts designed to test competing explanations of sound symbolism, but found no evidence for mediation via language, and only limited evidence for accounts based on multisensory integration and a general magnitude system. Instead, we suggest that the observed incongruency effects are likely to reflect phonological processing and/or multisensory attention. These findings advance our understanding of sound-to-meaning mapping in the brain.

Consistency of synesthetic associations over time is a widely used test of synesthesia. Since many studies suggest that consistency is not a completely reliable feature, we compared the consistency and strength of synesthetes’ grapheme-color associations. Consistency was measured by scores on the Synesthesia Battery and by the Euclidean distance in color space for the specific graphemes tested for each participant. Strength was measured by congruency magnitudes on the Implicit Association Test. The strength of associations was substantially greater for synesthetes than non-synesthetes, suggesting that this is a novel, objective marker of synesthesia. Although, intuitively, strong associations should also be consistent, consistency and strength were uncorrelated, indicating that they are likely independent, at least for grapheme-color synesthesia. These findings have implications for our understanding of synesthesia and for estimates of its prevalence since synesthetes who experience strong, but inconsistent, associations may not be identified by tests that focus solely on consistency.