Search Abstracts | Symposia | Slide Sessions | Poster Sessions
Decoding words during sentence production: ECoG reveals syntactic role encoding and structure-dependent temporal dynamics
Poster Session C, Saturday, September 13, 11:00 am - 12:30 pm, Field House
Adam Morgan1, Orrin Devinsky1, Werner Doyle1, Patricia Dugan1, Daniel Friedman1, Adeen Flinker1,2; 1NYU Grossman School of Medicine, 2NYU Tanon School of Engineering
Language production research has primarily focused on single-word tasks, such as picture naming. In contrast, the neural underpinnings of sentence production remain less studied due to experimental challenges and limitations of traditional neural measures (e.g., fMRI, MEG), which are sensitive to motor artifacts and offer limited spatial/temporal resolution. Even the assumption that isolated word production mirrors sentence contexts remains largely untested. To address this, we used electrocorticography (ECoG) to record brain activity in 10 awake neurosurgical patients during an overt production experiment. In a picture naming block, participants repeatedly named six characters (e.g., chicken, nurse). They then completed a sentence production block, describing cartoon scenes in response to questions in active (Who hit whom?) or passive (Who was hit by whom?) forms, resulting in active (“Dracula hit Frankenstein”) and passive (“Frankenstein was hit by Dracula”) sentences. Using machine learning classifiers, we identified neural patterns uniquely associated with each word during picture naming. Models were trained for each of seven ROIs, 20 time windows (-750 to 250ms from speech onset), and for each patient, resulting in ~1,400 classifiers. We validated these models using held-out picture naming trials, where they successfully decoded word identity in the time window that the model’s training data came from, consistent with feedforward models of lexical access. Next, to test generalizability, we used these classifiers to predict word identity throughout sentence production. For active sentences, we successfully decoded each word in the order of production – i.e., subject then object. Because the classifiers were trained only on picture naming data, this constitutes strong evidence for shared representations between single word production and sentence production. Intriguingly, applying this same procedure to passive sentences revealed a different pattern. While we again successfully decoded word identity with above chance accuracy, across models, both the subject and the object remained active throughout the entire sentence. This was driven by sustained representations in prefrontal cortex, which has previously been implicated in the processing of non-canonical sentences like passives, but its particular function has remained a topic of debate. Here, we show not only that it plays a role in sustaining word representations, but furthermore that it encodes words’ sentence position, with inferior frontal gyrus (IFG) selectively encoding subjects and middle frontal gyrus (MFG) objects. In contrast, sensorimotor cortex, which encodes articulatory information, patterned with active sentences, encoding words in the order in which they were produced. Our findings reveal a previously uncharacterized division of labor within the language network. Sensorimotor cortex encodes lexical information robustly and in a task-agnostic way, while prefrontal regions are sensitive to syntactic structure, likely reflecting flexibility under varying task demands. Furthermore, we unexpectedly uncovered a means by which the brain tracks syntactic roles: a spatial code whereby subjects are selectively encoded in IFG and objects in MFG. We propose that the complex temporal dynamics of word processing in prefrontal cortex may impose a subtle processing pressure over the course of language evolution, offering a possible explanation for why nearly all the world’s languages place subjects before objects.
Topic Areas: Language Production, Syntax and Combinatorial Semantics