Search Abstracts | Symposia | Slide Sessions | Poster Sessions
The neural network supporting sign language comprehension: A dynamic causal modeling study
Poster Session A, Friday, September 12, 11:00 am - 12:30 pm, Field House
Jieying He1, Brendan Costello1,2, Pedro M. Paz-Alonso1,2, Manuel Carreiras1,2,3; 1BCBL. Basque Center on Brain, Cognition and Language, Donostia, Spain, 2Ikerbasque, Basque Foundation for Science, Bilbao, Spain, 3University of the Basque Country (UPV-EHU), Spain
Sign language provides a unique and natural opportunity to explore the cognitive and neural mechanisms of language processing because its linguistic structure is coded in movement and space and decoded through vision. The neural network supporting sign language processing is poorly understood compared to spoken and written language. Using dynamic causal modeling (DCM), this study investigated the neuronal network underlying Spanish Sign Language (LSE) comprehension and compared the model with two other language modalities: written and auditory words. Three groups of hearing native Spanish-speaking participants with varying experience in LSE took part in the study: 23 proficient native signers, 20 proficient late signers, and 23 non-signers. Three modalities of stimuli were designed: sign stimuli (signs, pseudosigns, scrambled video as baseline), written stimuli (words, pseudowords, scrambled words as baseline), and audio stimuli (words, pseudowords, rotated speech as baseline). Participants viewed the stimuli while performing a simple detection task to maintain attention (i.e., indicate the direction of an arrow, presented in 10% of trials). Based on brain regions identified in the current study and previous DCM research on written and spoken language comprehension, we constructed a DCM model for each modality with six volumes of interest (VOIs) in the left hemisphere: inferior occipital gyrus (IOG), ventral occipital-temporal cortex (vOTC), inferior parietal lobule (IPL), posterior superior temporal sulcus (pSTS), anterior superior temporal sulcus (aSTS), and the pars orbitalis of inferior frontal gyrus (IFGorb). We used equivalent contrasts to determine the coordinates of the six VOIs across three modalities (signed, written, auditory): IOG was identified in the All>Null contrast (All includes words, pseudowords, and baseline); vOTC, pSTS, and IPL in the Word-like>Null contrast (Wordlike includes words and pseudowords); left aSTS and IFGorb in the Word>Baseline contrast. Similar to written and auditory words, sign language comprehension recruited multiple routes from the perceptual cortex (IOG) to higher-order semantic areas (IFGorb) such as IOG→vOTC→pSTS→IFGorb and IOG→pSTS→IFGorb. These two pathways suggest that semantic information (in the IFGorb) can be accessed from the visual cortex (in the IOG) via a route with or without vOTC region. In the sign language model, the pSTS region is the critical hub that transmits activations to other regions, reflecting crucial phonological processing involved in sign language comprehension. In contrast to a sublexical pathway (vOTC→IPL→pSTS) in written and auditory modalities, the reversed pathway (pSTS→IPL→vOTC) reflects specific roles of the left IPL (e.g., feedback activation during the precise configuration and location of hands in space that conform to phonological structure) in sign language processing. Additionally, the absence of bidirectional connectivity between the left aSTS and IFGorb suggests that sign language processing may involve alternative routes for semantic integration compared to the other two modalities. The most striking finding of this study is that the model for spoken or written language also accounts for the neural architecture underlying sign language processing, with multiple links between the processing of (visual) input with phonological or semantic processing. While the nodes are the same, the differences in connectivity provide new avenues for exploring how modality shapes language processing.
Topic Areas: Signed Language and Gesture, Multilingualism