Minimally invasive surgery and automated robot-guided surgery are likely to be performed under extreme conditions, which require new demands on intraoperative information acquisition capabilities. Three-dimensional (3D) brain point cloud generation plays an essential role in overcoming the visual limitations of these surgeries.

One of the applications of generative AI is in the field of 3D brain point cloud generation. By using deep learning algorithms, the 3D point cloud of the patient's brain can be generated for reconstruction during minimally invasive surgery. However, some conventional existing point cloud reconstruction methods still have limitations when applied directly to real surgical scenarios.

Now, a research team led by Prof. WANG Shuqiang from the Shenzhen Institute of Advanced Technology (SIAT) of the Chinese Academy of Sciences has proposed a hierarchical shape-perception network (HSPN) for better 3D brain reconstruction.

The study was published in IEEE Transactions on Neural Networks and Learning Systems on May 11.

The proposed framework of HSPN consists of an encoder, which has a predictor and multiple encoding blocks, and a corresponding decoder. The predictor is constructed to generate point clouds that accurately describe the incomplete images and then complete these point clouds with high quality.

Considering the unified information flow for communicating with adjacent modules, the researchers constructed a hierarchical attention pipeline to transmit local attention features between encoding and decoding blocks.

Meanwhile, they designed attention gate blocks (AGBs) in encoding and decoding blocks to efficiently aggregate local geometric features of incomplete point clouds transmitted by hierarchical attention pipelines and internal features of reconstructing point clouds. The employment of AGB could significantly increase the detailed expression ability while reducing generation error and enhancing stability.

With the proposed HSPN, 3D shape perception and completion could be achieved spontaneously. Comprehensive results measured by Chamfer distance and Point Cloud-to-Point Cloud (PC-to-PC) error demonstrated that the proposed HSPN outperformed other competitive methods in terms of qualitative displays, quantitative experiments, and classification evaluation.

"The proposed method has a significantly short inference time, which enables effective real-time feedback of local image properties," said Prof. WANG. "This feedback can guide doctors to find diagnostically valuable surgical locations."

Fig. 2 Comparison process of different point numbers of the predictor. (Image by SIAT)