Search Abstracts | Symposia | Slide Sessions | Poster Sessions
A common temporo-parietal network for verbal and non-verbal combinatorial semantic processing
Poster Session C, Saturday, September 13, 11:00 am - 12:30 pm, Field House
Gina Humphreys1, Matthew Lambon Ralph1; 1University of Cambridge
Successful time-extended semantic cognition requires the brain to integrate information continuously to form a coherent mental model of the world. The brain can build up and update a dynamic semantic representation (a “semantic gestalt”) as each new word or nonverbal stimulus is processed. The temporo-parietal network, specifically the superior temporal sulcus (STS) and temporo-parietal junction (TPJ) play a key role in combinatorial semantic processing in verbal tasks. For instance, this network shows increased activation for meaningful sentences vs. word lists, or meaningful sentences vs. non-meaningful sentences, or when reading a narrative that is congruent with a prior context vs. unrelated in meaning (Branzi, Humphreys et al., 2020; Humphries et al., 2006; Humphries et al., 2007; Vandenberghe et al., 2002; 2004). Combinatorial semantic processing occurs not only in verbal tasks but also in the non-verbal domain, such as when watching a silent movie. However, there has been relatively little research investigating whether verbal and non-verbal task engage a common or distinct system. Furthermore, outside of the domain of semantic cognition, a very similar temporo-parietal network is considered to part of a network specialised Theory of Mind (ToM) processing, that is the ability to understand that others have different thoughts and feelings to oneself (Carrington & Bailey, 2007; Shurz et al., 2021). This network alignment could be explained by: 1) a ToM confound in existing language studies since the majority of stimuli involve human participants; or 2) given that ToM tasks also involve computing a coherent time-extended semantic narrative, it is possible that these findings could be integrated into a more general “semantic gestalt” model. In the current fMRI study we investigated: 1) to what extent is the temporo-parietal network activated for combinatorial processing across domains, or whether it specialised for ToM tasks, and 2) to what extent does verbal and non-verbal combinatorial semantic processing rely on a common or distinct neural system. In a 2 x2 design participants were presented with semantically coherent vs. scrambled verbal narratives and non-verbal cartoon strips. In order to test the ToM hypothesis, half of the verbal narratives included ToM items, whilst the other half described events with no human participants (Deen et al., 2015; Dodell-Feder et al. 2011). The results showed a common bilateral temporo-parietal system for combinatorial semantic integration, with the entirety of the length of the STS/TPJ showing stronger activation for the coherent vs. incoherent items in both the verbal and non-verbal tasks. Bilateral STS and left TPJ showed overall stronger activation for the verbal vs. non-verbal task, whereas right TPJ was more active for the picture task but there was no interaction with coherence. Activation was found to be stronger for ToM > non-ToM items. However, the size of this difference was greatly reduced/eliminated when controlling for important semantic/linguistic properties which are known to influence the ease of semantic integration. Together these results provide evidence in favour of a temporo-parietal network for multi-modal combinatorial semantics.
Topic Areas: Syntax and Combinatorial Semantics,