Researchers led by Prof. XIE Chengjun and Associate Prof. ZHANG Jie at the Hefei Institutes of Physical Science of the Chinese Academy of Sciences have developed a novel deep learning-based method for satellite imagery.

Their method, called the Frequency-Adaptive Mixture of Experts Network (FAME-Net), has been accepted for publication in the 2024 Proceedings of the Association for the Advancement of Artificial Intelligence (AAAI).

High-resolution multispectral imagery is essential for agriculture, mapping, and environmental protection. However, direct acquisition of such images faces technological limitations. To overcome this challenge, pan-sharpening techniques combine high-resolution panchromatic (PAN) and low-resolution multispectral images. Recent advances in deep learning have improved spectral and spatial detail in pan-sharpening, but neural networks still struggle with frequency bias and adaptation to diverse remote sensing content.

Based on the discrete cosine transform and the Mixture of Expert concepts, the researchers proposed the FAME-Net, which uses a frequency mask predictor for adaptive high- and low-frequency masking.

Different expert networks process these frequency-specific features, allowing focused attention on different frequency ranges. FAME-Net dynamically adapts its masks to different image contents by integrating multiple expert outputs through a gating mechanism.

In comparative analyses with existing state-of-the-art methods, FAME-Net not only demonstrates superior performance in preserving spectral quality and enhancing spatial resolution but also shows excellent performance in full-resolution remote sensing imagery.

This study provides new insights into the field of image processing and demonstrates the effectiveness of integrating dynamic network structures and frequency domain information.

The overall structure of FAME-Net. (Image by ZHANG Jie)

The result of FAME-Net was compared against nine other methods on WorldView-III dataset. （Image by ZHANG Jie)