Search Abstracts | Symposia | Slide Sessions | Poster Sessions
The Role of Context Gating in Predictive Sentence Processing
Poster Session B, Friday, September 12, 4:30 - 6:00 pm, Field House
Yasemin Gokcen1, David Noelle1, Rachel Ryskin1; 1University of California Merced
Humans predict upcoming language input based on the preceding sentence context and their prior knowledge of the language and the world (Altmann & Mirkovic, 2009; Dell & Chang, 2014; Federmeier, 2007; Levy, 2008; Ryskin & Nieuwland, 2023, inter alia). For example, the N400 event-related potential (ERP) component is more negative in response to a word that is less predictable from the preceding sentence context compared to one that is more predictable (e.g., Frank et al., 2015; Kutas & Federmeier, 2000). The predictability of a word can be estimated using surprisal, the negative log probability of a word in context, from a language model (Levy, 2008; Hahn et al., 2022). However, human memory is imperfect. How do humans maintain the linguistic context for prediction over multiple timescales (e.g., immediately preceding words as well as the larger discourse) and optimize working memory resources such that elements that are most informative for prediction are maintained while others are not? Working memory processes associated with prefrontal cortex (PFC) have been proposed to perform such functions (O'Reilly et al., 1999) and have been modeled via neural networks with gating mechanisms which learn when to maintain important information and when that maintained information needs to be updated (Servan-Schreiber & Cohen, 1992; Hochreiter & Schmidhuber, 1997). Yet, past fMRI work suggests that the prefrontal regions associated with non-linguistic working memory (the multiple demand network; Fedorenko et al., 2013) are not meaningfully engaged during listening comprehension tasks (Blank et al., 2017, Diachek et al., 2020, Shain et al., 2019). To shed light on this, we collected EEG data while participants listened to stories from the Natural Stories corpus (Futrell et al., 2020) and extracted ERPs for each word in the stories. We replicate findings from previous literature that words that are low in predictability based on preceding context (high in surprisal) elicit larger N400 effects than predictable words (e.g., Frank et al., 2015, Michaelov et al., 2020, Szewczyk & Federmeier, 2022 ). To study how gating may play a role in next-word prediction, we use a performance difference metric between language models with and without gating, which we show is sensitive to word-by-word working memory demand. Long-short term memory (LSTM) networks include these PFC-like gating mechanisms while recurrent neural network (RNN) models do not. We derive a residual measure that captures the extent to which the LSTM models outperform the RNN models (Aurnhammer & Frank, 2019). We show that this residual measure is associated with the word-by-word engagement of WM (as operationalized by multiple theories), suggesting that a language model's ability to gate words in the context is particularly helpful in the same situations where WM appears to be taxed in humans. This tentatively suggests that the PFC structures which have been associated with gating (Braver & Cohen, 2000) may support adaptive memory for context during language comprehension. Finally, we explore the neural timecourse of these gating effects, finding that they appear to have most predictive power in a time window following the N400.
Topic Areas: Speech Perception, Computational Approaches