Chinese researchers have demonstrated a new paradigm for deep learning-based computational imaging (CI) that takes both physical model and training data into consideration. This enables them to obtain state-of-the-art results in single-pixel imaging (SPI).

This research was conducted by SITU Guohai from the Shanghai Institute of Optics and Fine Mechanics (SIOM) of the Chinese Academy of Sciences (CAS). Results were published in Photonics Research on December 13.

CI techniques mainly focuses on the design of encoding methods and decoding algorithms. Traditional strategy uses expert knowledge to design the encoding system (forward physical model), and then exploit the model driven optimization algorithms for decoding. Recent data-driven strategy intelligently designs the encoding mode and reverse decoding model by using training data, which greatly improve the imaging quality and efficiency. However, its practical applications are restricted by the difficulties in data acquisition, generalization and interpretability.

In this research, the team proposed a novel paradigm for CI, which makes fully use of the training data and the physical model. They demonstrated in SPI the training data was used to optimize the encoding patterns and the decoding network, while the physical model was utilized to eliminate the artifacts in the predictions due to the generalization issue of the train network model.

Their proposed method (fine tune) can obtain the best imaging fidelity among typical SPI algorithms including linear correlation and compressive sensing when the sampling ratio is as low as 6.25%.

They also built a SPI-LiDAR (Laser Detection and Ranging) system and verified the proposed method in outdoor experiments. A tower located at about 570 m away from the LiDAR system was successfully reconstructed by the proposed method, suggesting its great potential for remote sensing.

This work was supported by the Chinesisch-Deutsche Zentrum für Wissenschaftsforderung, the National Natural Science Foundation of China and the Key Research Program of Frontier Sciences of CAS. 

Schematic diagram of the physics enhanced deep learning approach for SPI. (a). The data-driven learning framework; (b). The SPI system; (c). The model-driven fine-tuning process. (Image by SIOM)

A comparison with typical SPI algorithm. (a-c) are simulation, laboratory and outdoor results, respectively. (Image by SIOM)