Search Abstracts | Symposia | Slide Sessions | Poster Sessions
The role of predictive processing in American Sign Language fingerspelling
Poster Session D, Saturday, September 13, 5:00 - 6:30 pm, Field House
This poster is part of the Sandbox Series.
Joseph Palagano1, Lorna Quandt1; 1Gallaudet University
Learning is often highly embedded within social interactions, and this prototypical form of learning may be conceptualized as a shared cooperative activity, as proposed by Bratman (1992), during which the interactive co-development of conceptual or situational models aids knowledge-sharing. Advancements in the methodologies and modeling of neurobiology open the field to asking new questions about interactive learning, specifically through its most ecologically valid modality: communication. One such methodology, hyperscanning, also known as dual-brain scanning, allows for analyzing spatio-temporal interbrain relationships between two interacting participants. Successful communication demands that we move beyond merely encoding and decoding utterances and carefully attempt to align linguistic proclivities (e.g., rate of production & attention) and content (e.g., predictability & register) with our partners’ intent, knowledge, and emotions (Snedeker & Trueswell, 2003; Verga & Kotz, 2013; Doyle & Frank, 2016). Resulting in alignment: an increased similarity of behaviors among communicating dyads. Prior findings suggest that neural activation patterns converge across participants as behavioral alignment increases (Menenti et al., 2012). However, it remains unclear what cognitive mechanisms support this neural coupling and how it relates to behavioral similarities observed, particularly, how plastic such an effect may be, and if findings persist in signed languages, much like in spoken languages. To address this, we propose a dual-electroencephalography (EEG; 64-channel) hyperscanning study to assess real-time phonologic alignment during American Sign Language (ASL) fingerspelling. We hypothesize, in line with prior models of fingerspelling, that both top-down and bottom-up predictive processing mechanisms influence fingerspelling perception and production, as reflected in behavioral and neural alignment. We will recruit signing adult dyads (N = 30 dyads) from Gallaudet University. Dyads will take turns producing phonologically predictable and unpredictable fingerspelling segments and matching their partner's production to a set of potential responses (e.g., predictable: cheerios, churros, cheezits, cheetos; unpredictable: chanrkes, cholus, chemotes, chitres). This task was chosen to increase feelings of cooperation and goal-driven behavior, known to prompt alignment. Participants will also complete background surveys and assess their partner’s fluency and likability. While performing the task described above, we will record electroencephalography (EEG) signals from a single 64-channel BrainVision cap positioned to fit the 10-20 system split evenly between the participants (i.e., 32/participant; Jasper, 1958; Douglas et al., 2023). Neural alignment will be assessed via a wavelet coherence analysis–with specific interest in the alpha, theta, and beta oscillatory bands (Pérez et al., 2019)–which provides a measure of cross-brain synchrony, i.e., time-locked neural events between specific regional pairs. This will result in a measure of alignment assigned to an electrode pair across participants (i.e., each participant’s Cz), allowing us to assess differences due to role (i.e., fingerspeller/perceiver), fluency (i.e., skilled fingerspellers/novice), and condition (i.e., predictable/unpredictable). We predict that all dyads will show more neural alignment during the predictable condition. If neural alignment is a neural proxy for mutual intelligibility, we expect dyads to perform more accurately as wavelet coherence increases, regardless of condition. Preliminary data analysis from the hyperscanning paradigm will be shared at the 2025 SNL meeting.
Topic Areas: Signed Language and Gesture, Methods