Search Abstracts | Symposia | Slide Sessions | Poster Sessions
Language as “assistive technology” for programming: perisylvian language network encodes meaning of Python programming algorithms
Poster Session E, Sunday, September 14, 11:00 am - 12:30 pm, Field House
Yun-Fei Liu1, Marina Bedny1; 1Johns Hopkins University
Computer programming is a recent cultural invention without specific neural adaptations. A key outstanding question is whether the language system is “recycled” by programming, and if so, what role does it play? We and others have shown that the language system is not robustly active during code comprehension. Instead, fronto-parietal reasoning systems are active when programming experts understand and write code. However, despite low activation levels, some studies found that multivariate neural patterns in perisylvian language cortices distinguish between different “control structures” (i.e., “for” loops and “if” conditionals). Here, we aim to replicate and extend this effect. Specifically, we asked whether the language network encodes format-independent representations of algorithms. We recruited Johns Hopkins undergraduate students (n=30) who completed one semester of introductory Python course. During the scan, students read either Python code or “pseudocode” passages. The pseudocode passages had similar algorithmic meanings to Python code but were written in plain English. After each stimulus presentation, participants got an input to the algorithm and a potential output, and participants judged whether the output was correct. Participants also performed matched code and pseudocode memory control tasks. We used independent functional localizers to identify the language and reasoning networks in individual participants. Univariate analysis replicated previous findings that Python code reading engaged the reasoning much more so than the language network. Pseudocode also produced stronger activity in the reasoning network relative to its control task. Despite the overall low activation in the language network, we found that neural responses to both code and pseudocode peaked earlier in the language network than in the reasoning network. Moreover, in multivariate pattern analysis (MVPA), we observed significant decoding of "for" versus "if" algorithms in individual-subject language-selective regions of the lateral temporal cortex, for both code and pseudocode. Whole-brain searchlight analyses further showed cross-decoding (trained on code and tested on pseudocode, and vice versa) in two regions associated with high-level language processing: the superior temporal sulcus (STS) and angular gyrus (AG). These findings indicate that the language network may represent the meanings of programming algorithms in a format-invariant manner. Building on these findings, we propose that the language network may generate early, abstract “gist” representations of Python code algorithms. These “gist” representations may facilitate subsequent detailed processing in reasoning regions.
Topic Areas: Computational Approaches, Syntax and Combinatorial Semantics