Search Abstracts | Symposia | Slide Sessions | Poster Sessions
Interpreting Neural Signatures of Word and Nonword Processing Across Speakers and Dialects using SVMs and RSA
Poster Session E, Sunday, September 14, 11:00 am - 12:30 pm, Field House
Seerat Sidhu1, Martin Oberg2, Alexis Black2; 1McGill University, 2The University of British Columbia
Background Decades of behavioural research have shown that word recognition is an incremental process involving competition among multiple lexical candidates. Recent work by McMurray et al. (2022) demonstrated that SVM-based machine learning can decode the neural spatiotemporal encoding of phonetically similar words and non-words from EEG signals, and that decoding response patterns closely mirror prototypical lexical competition effects. Here we describe two studies that replicate and extend McMurray et al. (2022). In Study 1, we decode EEG signals associated with word and nonword recognition within a lexical competition framework. In Study 2, we extend this paradigm to decode across different speakers, dialects, and genders. In both, we further assess the decoder’s sensitivity to individual differences by correlating decoding performance with behavioral task data, and explore spatio-temporal differences in word-form representations using representational similarity analysis (RSA). Methods: Study 1 20 participants were exposed to 4 word and 4 non-word pairs. Pairs were constructed such that the onsets were identical (e.g. “badger” and “baggage”), thus prompting lexical competition. Participants heard words and nonwords in pseudo-random order for a grand total of 960 trials. Data were acquired with a 32-channel Brain Vision Actichamp Plus. Study 2 20 participants were exposed to a new set of 4 word and 4 non-word pairs selected from the Auditory English Lexicon Project corpus hosted by the National University of Singapore. Word and non-word tokens are produced by 6 speakers, which are further characterized as falling into three dialects; each dialect is produced by 1 male and 1 female speaker. Data were acquired with a 128-channel, MagStim EGI. An SVM was trained to predict word-form pairs from one another, as well as speaker identity and dialect identity. The time-course of target vs cohort vs unrelated identification is compared across word and non-words in both studies. In addition, across both studies within-participant Representational Dissimilarity Matrices (RDMs) were constructed for each decoding task (e.g., word identity, speaker identity) using pairwise Euclidean distances between activation patterns across electrodes and timepoints. These participant-specific neural RDMs were compared with one another to assess the similar structure of neural representations underlying word and non-word processing across time between participants. Results: Study 1 replicates McMurray et al (2022). Target word-forms were identified significantly above chance (chance = 0.12), and the time-course of auditory integration mirrors findings from the eye-tracking psycholinguistic literature. Study 2 demonstrates the same pattern of word-form identification and competition dynamics across speakers and dialects, and rapid successful identification of speaker identity, dialect, and gender with very little evidence of confuseability. Individual difference analyses are ongoing. Conclusion: Our findings confirm the robustness of McMurray et al. (2022)'s results, validating their reproducibility and highlighting machine learning's efficacy in neural decoding for spoken word recognition. Further implementation of RSA will help clarify the structure of neural representations over time, enhance interpretability of decoding models, and provide a framework for comparing individual and group-level representational dynamics. References: McMurray et al. (2022). Decoding the temporal dynamics of spoken word and nonword processing from EEG. NeuroImage, 260, 119457.
Topic Areas: Speech Perception,