A neurocircuit loop in multimodal semantic integration was recently found by a research team led by Dr. DU Yi from the Institute of Psychology of the Chinese Academy of Sciences (CAS) using non-invasive chronometric Transcranial Magnetic Stimulation (TMS) technique.

For the first time, by revealing temporal dynamics of the frontal lobe control and temporal representation regions, the researchers provided new insights into dynamic brain networks of multimodal semantic processing.

Semantic processing is a defining feature of human cognition. Semantic cognition includes at least two interactive components: a long-term storage of semantic knowledge supported by posterior middle temporal gyrus (pMTG) and control processes supported by inferior frontal cortex (IFG).

The interaction of control processes with conceptual representations allows us to flexibly retrieve knowledge. However, it remains unknown how the control region dynamically interacts with the temporal representation region during semantic processing.

By splitting the possible gesture-speech integration period into short time windows and applying inhibitory double-pulse TMS over pMTG or IFG separately, the researchers observed differential time-selective disruption of the semantic congruency effect (i.e., a cost of reaction time due to semantic conflict between gestures and speech words).

A two-stage gesture-speech integration neurocircuit, pMTG-IFG-pMTG, was thus proposed. Specifically, in the pre-lexical stage (i.e., before lexical retrieval), there is a pMTG-to-IFG sequential involvement for activating gesture semantics and top-down constraining the phonological processing of speech.

In the post-lexical stage (i.e., after lexical retrieval), there is an IFG-to-pMTG feedback signal, might be implicated for the control of goal-directed semantic representations and multimodal semantic unification.

This study further deepens the current knowledge and inspires future investigation of the temporal dynamics and cognitive processes of the amodal semantic network.

This work entitled "TMS reveals dynamic interaction between inferior frontal gyrus and posterior middle temporal gyrus in gesture-speech semantic integration" was published in Journal of Neuroscience on Nov. 16, and funded by grants from the National Natural Science Foundation, the Scientific Foundation of Institute of Psychology of CAS, and the Strategic Priority Research Program of CAS.

The proposed two-stage integration circuit in semantic processing of gestures and speech. (Image by Dr. DU Yi)