A research team led by Prof. Cao Jianzhong from the Xi'an Institute of Optics and Precision Mechanics (XIOPM) of the Chinese Academy of Sciences (CAS) recently proposed an image fusion network based on variational auto-encoder (VAE). Their results wwere published on Infrared Physics and Technology.
Infrared and visible image fusion mainly aims at combining infrared radiant features with visible textural features into one image, which contains diverse meaningful information for subsequent processing tasks.
Some recent methods are based on auto-encoder (AE), which has the ability of restoring the original image. However, if some random changes happen to the feature vectors, the reconstructed image will be obscure or even meaningless. The reason is that some feature vector values have greatly changed from their initial values. It means that AE is not very suitable for this task. Different from AE, VAE can reconstruct images which has similar radiation and detail features to the original image.
Based on the abovementioned observation and analysis, CAO and his team proposed a new fusion network, which for the first time applied VAE to infrared and visible image fusion, meeting the requirement for generating a new image from two source images. The whole framework was disassembled into image fusion network and infrared feature compensation network.
According to the experiments results, the proposed method outperforms other traditional and deep learning methods under different evaluation metrics. In the future, and it will provide a guide for designing novel framework for infrared and visible image fusion.
Structure diagram of infrared and visible image fusion network. (Image by XIOPM)
52 Sanlihe Rd., Xicheng District,
Beijing, China (100864)