Huashan Researchers Publish in Nature Neuroscience: Shared and Distinct Neural Encoding Mechanisms Underlying Imagined and Overt Speech | Huashan Research

When we silently rehearse a sentence in our minds without producing any sound, how does the brain generate this “silent speech”? This question concerns a longstanding unresolved problem in neuroscience. Although the cognitive processes underlying overt speech have been studied relatively systematically, direct evidence of the internal neural mechanisms supporting imagined speech remains limited. Neurolinguistic models based on noninvasive functional mapping suggest that imagined and overt speech may share certain motor planning processes and patterns of frontoparietal activation. However, the spatial and temporal resolution of noninvasive techniques has limited direct investigation of their encoding relationships at finer spatiotemporal scales.

Supported by China’s Science and Technology Innovation 2030—“Brain Science and Brain-Inspired Research” Major Project, researchers led by Wu Jinsong and Lu Junfeng at Huashan Hospital, Fudan University, and the Shanghai Key Laboratory of Brain–Computer Interface Clinical Trials and Translation, together with Li Yuanning’s team at ShanghaiTech University, Chen Gao’s team at the Second Affiliated Hospital of Zhejiang University School of Medicine, and collaborators, used high-density electrocorticography (ECoG) to systematically characterize shared and distinct neural encoding patterns for articulatory movement trajectories during imagined and overt speech. Building on these findings, the researchers developed a deep learning decoding framework to generate text and speech and reconstruct articulatory movement trajectories from brain activity. The study, entitled A neural architecture for imagined and overt speech motor dynamics, was published online in Nature Neuroscience on September 29, 2026. Overt and Imagined Speech: Distinct Spectral Contributions Across Speech Conditions Using high-density intracranial electrocorticography (ECoG), the researchers recorded frontoparietal neural activity while participants performed imagined speech tasks, termed Speech Imagery (SI), and overt speech tasks, termed Speech Articulation (SA). The two tasks differed in the frequency bands that contributed most strongly to articulatory motor encoding. During overt speech, the high-gamma band (70–150 Hz) made the largest contribution, followed by the beta-1 band (12–24 Hz). During imagined speech, however, the lower-frequency beta-1 band predominated. These findings suggest that the brain relies on different spectral components when executing an overt action and internally simulating it. High-gamma activity contributed primarily to articulatory motor execution, whereas beta-1 activity may participate more broadly in coordinating speech networks involved in motor planning, internal simulation, and the execution and maintenance of articulation.

The Beta-1 Band: Shared Upstream Speech Planning and Distinct Downstream Processes The researchers then focused on the beta-1 band, which predominated during imagined speech, and performed detailed functional classification and spatiotemporal analyses across the recorded electrodes. They found that the middle premotor cortex (mPMC), subcentral gyrus (subCG), and postcentral gyrus–supramarginal gyrus junction (POCG–SMG) exhibited shared spatiotemporal encoding patterns during imagined and overt speech. These regions integrated motor plans involving multiple articulators, including the lips, tongue, jaw, and larynx. By contrast, neural populations in the primary ventral sensorimotor cortex (vSMC) showed pronounced specificity to the speech condition. Some populations were involved in executing movements of individual articulators, while others were involved in internally simulating articulatory movements. These populations were spatially interleaved. Further temporal analyses showed that activation of the shared clusters preceded that of the condition-specific clusters during both overt and imagined speech. The analyses also indicated a consistent direction of information flow across the two conditions: signals originated in the parietal POCG–SMG region, propagated anteriorly to the frontal mPMC and subCG, and ultimately converged on condition-specific neural populations in the vSMC to support either articulatory execution during overt speech or internal simulation during imagined speech.

One Framework, Bidirectional Decoding: Cross-Condition Reconstruction of Imagined Speech Based on these findings, the researchers developed a decoding framework capable of classifying syllables, synthesizing speech, and reconstructing articulatory movement trajectories from the same segment of neural activity. During imagined syllable production, median syllable classification accuracy reached 80.4%. Speech synthesis and articulatory movement reconstruction also performed significantly better than chance-level baselines. Performance on all three principal measures was comparable to that achieved when decoding overt speech.

The model also demonstrated a degree of transfer between speech conditions. By exploiting neural representations shared by overt and imagined speech, a model trained on overt speech data could directly decode imagined speech, and transfer in the reverse direction was also possible. These findings provide a theoretical basis and suggest potential implantation targets for speech brain–computer interfaces that operate across the two conditions. For syllables not included in training, the model could also predict articulatory movement trajectories from ECoG signals. This offers a theoretical and engineering foundation for improving the generalizability and robustness of future imagined-speech brain–computer interface systems in clinical settings. Professor Wu Jinsong and Associate Professor Lu Junfeng of Huashan Hospital, Fudan University; Principal Investigator Li Yuanning of ShanghaiTech University; and Professor Chen Gao of the Second Affiliated Hospital of Zhejiang University School of Medicine are co-corresponding authors of the paper. Dr. Zhao Zehao of Huashan Hospital, Fudan University; Dr. Wang Zhenjie of ShanghaiTech University; and Dr. Liu Yan of Huashan Hospital, Fudan University are co-first authors. The study received substantial support from the Academic Committee of the Shanghai Key Laboratory of Brain–Computer Interface Clinical Trials and Translation. Funding was provided by China’s Science and Technology Innovation 2030—“Brain Science and Brain-Inspired Research” Major Project, the National Natural Science Foundation of China, the Shanghai Municipal Education Commission’s Research and Innovation Program, and other funding programs.

Original article by: Zhao Zehao, Department of Neurosurgery
Original article reviewed by: Lu Junfeng, Department of Neurosurgery
Original source: Official WeChat subscription account of Huashan Hospital, Fudan University
 Translation and republication notice: This article is an English translation of the Chinese original cited above. Copyright remains with the original authors and relevant rights holders. It is shared to communicate research developments and does not imply endorsement of this account or its services by the original authors, research teams, or affiliated institutions. The findings apply to the research conditions described and should not be interpreted as evidence of an approved clinical application or as a guarantee of benefit for individual patients.