Search Abstracts | Symposia | Slide Sessions | Poster Sessions
Structure building is more incremental during comprehension of spontaneous speech than during audiobook listening
Poster Session B, Friday, September 12, 4:30 - 6:00 pm, Field House
Cas W. Coopmans1,2, Laura Giglio2,3, Peter Hagoort2; 1New York University, 2Donders Institute for Brain, Cognition, and Behaviour, 3University of South Carolina
The naturalistic revolution in cognitive neuroscience has led to increased interest in the study of language use in naturalistic contexts. However, previous studies have primarily focused on audiobook listening, which is not the most natural mode of language use. Spontaneous speech, in contrast, comes closer to how language is used in daily life, and it has a number of features that affect the neurobiology of sentence processing, including incomplete sentences, disfluencies, and restarts. While neural signatures of syntactic structure building are found with both types of stimuli, the incrementality of structure building is found to be modulated by properties of the input across different studies. Against this background, we compare neural activity for structure building when people listen to coherent audiobooks versus spontaneous speech, in order to investigate how people’s sentence-processing strategy is affected by the properties of the linguistic input. In a within-subjects design, 30 Dutch-speaking participants listened to 25-minute audiobook stories and to 25-minute recordings of a person spontaneously answering general questions (e.g. “What is your favorite food?”) while their brain activity was measured with fMRI. The syntactic structures of the sentences in both the audiobooks and the spontaneous recordings were semi-automatically annotated by a constituency parser for Dutch. We then used the syntactic structures to quantify structure building at each word via an incremental ‘top-down’ strategy, which builds syntactic nodes at phrase opening, and a less incremental ‘bottom-up’ strategy, which builds nodes at phrase closing. Using forward encoding models, these metrics were regressed against fMRI time courses in three brain regions linked to sentence-level processing: the left inferior frontal gyrus (LIFG), posterior temporal lobe (LPTL) and anterior temporal lobe (LATL). All models included the control regressors sound power, word onset, frequency, surprisal and entropy, and model fit was quantified via linear mixed-effects models. Spontaneous speech elicited stronger brain activity than audiobooks, suggesting overall higher effort and/or engagement for comprehension of spontaneous speech. Moreover, the results showed a striking asymmetry between the effects of top-down and bottom-up node count: the top-down regressor significantly captured increased activity in the LIFG and LPTL, which was larger in spontaneous speech than in audiobooks. The significant effect of bottom-up node count, in contrast, was consistently negative in all ROIs and in both types of stimuli, suggesting that brain activity decreased at phrase closing. Taken together, the data suggest that the neural response to syntax is strongly driven by phrase-opening words, which allow structure to be built incrementally. This effect was enhanced for spontaneous speech, suggesting that structure building is more incremental during comprehension of spontaneous speech. More generally, the results confirm the idea that the type of linguistic input has an effect on the mode of syntactic processing, in line with previous studies showing language-specific differences in processing strategy. This result has important implications for the neurobiology of language, because it suggests that experimental findings might not be the same when more natural and ecologically valid input is used.
Topic Areas: Syntax and Combinatorial Semantics,