Search Abstracts | Symposia | Slide Sessions | Poster Sessions
Stereoelectroencephalography reveals neural signatures of multisensory integration in the human superior temporal sulcus during audiovisual speech perception
Poster Session A, Friday, September 12, 11:00 am - 12:30 pm, Field House
John Magnotti1, Yue Zhang2, Xiang Zhang1, Zhengjia Wang1, Yingjia Yu1, Kathryn Davis1, Sameer Sheth2, Isaac Chen1, Daniel Yoshor1, Michael Beauchamp1; 1University of Pennsylvania, 2Baylor College of Medicine
Human speech perception is multisensory, integrating auditory information from the talker's voice with visual information from the talker's face. Studies using BOLD fMRI have implicated the superior temporal gyrus (STG) in processing auditory speech and the superior temporal sulcus (STS) in integrating auditory and visual speech, but as an indirect hemodynamic measure, fMRI is limited in its ability to track the rapid neural computations underlying speech perception. We directly recorded neural activity in the STG and STS across 42 epilepsy patients implanted with stereoelectroencephalograpy (sEEG) electrodes. Patients identified single words presented in auditory, visual and audiovisual formats with and without added auditory noise. Seeing the talker's face provided a robust perceptual benefit, improving perception of noisy speech in every participant. Neurally, a subpopulation of bimodal electrodes concentrated in mid-posterior STG and STS showed short latency responses to auditory speech (71 ms) and visual speech (109 ms). Significant multisensory enhancement was observed in STS bimodal electrodes: compared with auditory-only speech, the response latency for audiovisual speech was 40% faster and the response amplitude was 18% larger. In contrast, STG bimodal electrodes showed neither faster multisensory latencies nor response enhancement. Overall, auditory-only speech responses were fastest in the STG while audiovisual speech responses were fastest in the STS. Significantly faster responses for audiovisual speech suggest the possibility that seeing the face of the talker may increase the speed, as well as the accuracy, of speech perception.
Topic Areas: Speech Perception, Multisensory or Sensorimotor Integration