Search Abstracts | Symposia | Slide Sessions | Poster Sessions
Slide Session B
Saturday, September 13, 10:00 - 11:00 am, Elstad Auditorium
Talk 1: Using a comma to investigate the node-tracking framework for the construction of phrase structure representations in the human brain
Matthew Nelson1, Christophe Smith, Bryant Barrentine, Marshall Holland, Bentley Nicole, David Basilico; 1University of Alabama at Birmingham
Much of what is described in the literature as models of sentence processing in the brain are essentially just different authors’ views on what parts of the brain are involved in what aspects of sentence processing. There is a dearth in the literature of mechanistic models capturing how populations of neurons themselves might enact sentence comprehension. In 2017, we introduced the node-tracking framework to address this. We derived this from observations that while reading a sentence, population-level neural activity measured by high gamma power (70-150 Hz) in the left Middle Temporal Gyrus (MTG) builds up word-by-word within a phrase but declines sharply after the phrase boundary. Concomitantly, left Inferior Frontal Gyrus (IFG) activity increases transiently only at phrase boundaries, in proportion to the amount of decline in MTG activity. The node-tracking framework posits that this MTG activity pattern reflects the maintenance of open-nodes in working memory, that is, the maintenance of the nodes of a tree structure representation of a sentence open to syntactic operations while that tree structure is being built. The IFG pattern is posited to reflect the number of nodes closing, which, assuming a Minimalist Grammar, reflects the degree of Merging occurring at a given moment. This framework has been highly impactful, yet some in the field question it. A key barrier of studies following our introduction of the framework are uncontrolled word-level parameters (e.g. word frequency, etc.), which obfuscate whether reported effects are consistent with the framework or merely reflect the effect of these parameters. To overcome this barrier, we recorded intracranial activity (sEEG and ECoG) in 20 epilepsy patient volunteers while they read sentences one word at a time (RSVP) and answered questions about the sentences. We manipulated the phrase structure in matched sentence pairs that they read, as in: (1a) While the man hunted, the deer ran into the woods. (1b) While the man hunted the deer, the rabbit ran into the woods. These sentences begin with identical sequences of words up to and including the target noun phrase ‘the deer’, which, through comma placement, is the subject of the main clause in (1a) but the direct object of the subordinate clause in (1b). The node-tracking framework makes clear predictions about the neural activity differences expected between 1a and 1b, which we find support for in the left MTG and IFG both within and across patients in our sample. Moreover, we observed marked ramping up activation in left MTG over the course of the opening phrases of the sentences, which is commonly reported in intracranial sentence processing studies across several groups. This ramping up is entirely explained by and consistent with open-node-related activation predicted by the node-tracking framework, thus this framework provides a parsimonious explanation for this phenomenon. Altogether, this study fruitfully advances the debate about the node-tracking framework, leading towards a presently lacking mechanistic understanding of how neuron populations enact sentence comprehension. These results may also inform future Brain-Computer Interface (BCI) treatments for communication at the sentence level.
Talk 2: A transmodal working memory area in precentral gyrus at the intersection of expressive and receptive language areas
Tyler Perrachione1, Rebecca Belisle1, Terri Scott1; 1Boston University
Diverse investigations into the neurobiology of language have identified a mid-precentral (PrCG) or posterior middle frontal (pMFG) region that is variously attributed to high-level receptive language (Lipkin et al., 2022), speech motor control (Belyk et al, 2021), pitch control (Hickok et al., 2023), working memory (WM; Scott & Perrachione et al., 2019), or grammatical expressive language (Rogalski et al., 2014; Silva et al., 2022). However, because this work is often based on whole-brain group averages or only one construct, it is unclear whether PreCG-pMFG contains only a single, undifferentiated language area, or whether there is more granular areal organization in individual brains. Here, we used two approaches to functionally characterize speech, language, and WM areas in PreCG-pMFG in individual brains: (i) a task-specific, hypothesis-driven function region of interest (fROI) analysis and (ii) a multi-task, hypothesis-free clustering analysis. Using individual task-specific activation maps on the cortical surfaces of N=25 young adults with typical speech, language, and hearing, we performed these analyses in an anatomically circumscribed parcel intended to broadly encompass portions of PreCG and pMFG known to be responsive to receptive or productive language. In the hypothesis-driven fROI approach, we delineated areas most strongly activated by a receptive language localizer (listening to intact vs. degraded speech; Scott et al., 2017) and, independently, a nonword repetition task (4-syllable vs. 1-syllable; Scott & Perrachione, 2019). From these, we characterized vertices as belonging either solely to the receptive language fROI, solely to the nonword repetition fROI, or common to both fROIs. In the hypothesis-free clustering approach, we performed k-means clustering across vertices based on fMRI effect sizes from 6 language and WM tasks (receptive language localizer, nonword repetition, real word repetition, nonword discrimination, visuospatial sequencing, and auditory digit span). The resultant 7 clusters had distinct fMRI task response profiles across the 6 tasks, including 3 clusters principally responsive to (i) receptive language (language localizer task), (ii) speech (word and nonword repetition tasks), and (iii) transmodal language/WM (all 6 tasks). Notably, these 3 clusters were spatially consistent with the 3 areas delineated in the fROI approach, suggesting that the productive-receptive overlap region (fROI approach) is largely the same as the region with uniquely heightened responses for all 6 tasks (clustering approach). Taken together, the two analyses provide converging evidence for a tripartite speech-language-working memory arealization, in which we find, consistently across subjects, (1) a posterior speech-specific region, (2) an anterior language-specific region, and (3) an intervening transmodal WM area. This transmodal WM area appears to be strongly responsive to both receptive language and WM tasks, regardless of whether they are verbal (both meaningful and meaningless) or spatial. Finally, we examined whether these areas were neuroanatomically differentiated based on their relative intracortical myelin content (T1w/T2w MRI; Glasser & Van Essen, 2011). The transmodal WM area was characterized by low intracortical myelin, despite lying directly adjacent to the highly myelinated speech-specific cluster in primary motor cortex. These results expand our understanding of the granularity of functional neuroanatomy in precentral areas for language and cognition.
Talk 3: A common framework for semantic memory and semantic composition
Ryan M.C. Law1, Olaf Hauk1, Matthew A. Lambon Ralph1; 1MRC Cognition and Brain Sciences Unit, University of Cambridge,
How the brain constructs meaning from individual words and phrases is a fundamental question for research in semantic cognition, language and their disorders. These two aspects of meaning are traditionally studied separately, resulting in two large, multi-method literatures, which we sought to bring together in this study. Not only would this address basic questions of how semantic cognition operates, but also because, despite their distinct focuses, both literatures ascribe a critical role to the anterior temporal lobe (ATL) in both aspects of semantics. Given these considerations, we explored the notion that common neurocomputational principles underlie the representing and constructing of semantic representations from words and word sequences, respectively. The ATL has been implicated in semantic memory (Rogers et al., 2004; Patterson et al., 2007; Lambon Ralph et al., 2017) and semantic composition (Coutanche et al., 2019; Pylkkänen, 2019, 2020; Călinescu et al., 2023), with evidence from neuropsychology, neuroimaging and computational modelling. Does the overlap in functional neuroanatomy hint at shared computations across the two systems? A neural network model by Hoffman et al. (2018) offers a unifying framework. The model takes word sequences as input and predicts both the upcoming words and each concrete word’s sensory-motor properties. In doing so, the model acquires representations that reflect the multimodal knowledge of each concept (McClelland & Rogers, 2003; Patterson et al., 2007; Lambon Ralph et al., 2017) while also accounting for co-occurrence and information integration over time (McClelland et al., 1989; St. John & McClelland, 1990; Rabovsky et al., 2018). By assimilating evidence from these different strands, we propose that the ATLs provide a unified function essential to semantic representation and thus play a central role in both literatures. If so, then adjective-noun processing should be sensitive to conceptual variables from semantic memory. We integrate one such variable, concreteness (e.g., lettuce/fiction), which, to our knowledge, has been underexplored in adjective-noun processing. We also explored the processing of denotation semantics, introduced by subsective vs. privative adjectives (e.g., bad vs. fake), which significantly modulates interpretation (Kamp, 1975; Partee, 2010; Del Pinal, 2015; Fritz & Baggio, 2020). CONTROL: xtvq lettuce/fiction SUBSECTIVE PHRASE: bad lettuce/fiction (implies “this is a lettuce”) PRIVATIVE PHRASE: fake lettuce/fiction (implies “this isn’t a lettuce”) We recorded neural activity using electroencephalography and magnetoencephalography (EEG/MEG) as thirty-six participants silently read words and phrases in RSVP (total n=600) and answered occasional comprehension questions (n=60). Source-localized region-of-interest analyses show that the left, and to a lesser extent the right, ATLs responded more strongly to phrases, irrespective of concreteness. Decoding analyses further revealed a time-varying representational format for adjective semantics, whereas representations of noun concreteness were more stable and maintained for around 300 ms. Further, the neural representation of noun concreteness was modulated by the preceding adjectives: decoders learning concreteness signals in single words generalized better to subsective than privative phrases. Together, our results suggest that the ATLs provide a unified function for semantic memory and composition. The present framework paves the way for understanding complex meaning construction in time-extended verbal behaviours, such as narratives.
Talk 4: Shared Neural Filters Reveal Modality-Invariant Semantic Representations in Intracranial Recordings
Aditya Singh1, Elliot Murphy2, Tessy Thomas3, Nitin Tandon4; 1UT Health
The human brain’s ability to extract the same meaning from different linguistic inputs (e.g., spoken versus written cues) suggests the presence of modality-invariant semantic representations. However, how such shared meaning is dynamically encoded in neural activity remains poorly understood. To address this, we recorded stereotactically placed depth electrodes (sEEG) from 15 neurosurgical patients performing two naming-to-definition tasks, in which target concepts were elicited by either orthographic or auditory cues (orthographic trials: mean = 370, s.d. = 150; auditory trials: mean = 413, s.d. = 104). We applied non-negative matrix factorization (NNMF) to trial-aligned high-gamma activity (70-150Hz), extracting low-rank spatiotemporal components that captured consistent functional groups aligned to distinct temporal segments. Clustering these trajectories revealed subject-agnostic task-related dynamics with certain stages invariant to both modalities. Using features from the NNMF components, a classifier could predict the named concept at accuracies significantly above chance (38%/26% vs 3% chance, p<0.001 paired t-test with FDR correction) for written and auditory task conditions, respectively. Using components with a sustained response during the cue period, we achieved significant pre-articulatory decoding (17% vs 3% chance, p<0.01) performance of both the word and the semantic concept (45% vs 20% chance, p<0.05). We trained a component-aligned decoding model using a transfer learning framework to assess cross-subject generalization. Each subject’s NNMF-derived spatiotemporal features were encoded with a subject-specific recurrent head, then mapped into a shared latent space. A shared decoder was trained on a source subject and then frozen, while new subject-specific encoders were trained to align to its latent space and decode target words. This approach allows for direct evaluation of cross-subject and cross-modality generalization while controlling model complexity. Additionally, to evaluate how neural activity tracks stimulus semantics, we compressed contextual language model embeddings into discrete codebooks using a vector-quantized variational autoencoder (VQ-VAE), producing compact, modality-agnostic targets representing meaning at a phrasal level. These enhancements revealed that shared latent dynamics across NNMF-aligned features support robust semantic transfer across individuals and modalities. By aligning interpretable neural trajectories across subjects, our framework enables scalable, cross-subject semantic decoding—advancing neuroprosthetic systems toward decoding intended meaning, not just speech output. This work shifts the framing of language decoding from “sound with meaning” to “meaning with sound.”