Researchers from the Changchun Institute of Optics, Fine Mechanics, and Physics of the Chinese Academy of Sciences developed a hybrid-level, dual-input Mamba framework to detect objects in fused visible and infrared images. This method combines feature fusion and decision-level refinement. It achieved a mean average precision (mAP) of 81.74% at the standard intersection-over-union threshold and 56.62% on average across the full range of thresholds on the DroneVehicle benchmark.

Tests on two public datasets also demonstrated competitive performance across various viewpoints, object types, and lighting conditions. The findings were reported in Pattern Recognition on September 10.

Visible images carry texture, edges, and scene details. Infrared images preserve object contours in low light. Together, they provide more complete information, but fusion does not automatically improve detection. Existing methods often blend the streams before state-space processing, introducing complementary cues, repeated information, and conflicting responses into the same representation. Once these cues are combined, the model cannot easily determine which to trust. This is important at night, around occluded objects, and in cluttered scenes, where the quality of the two inputs can differ significantly.

The proposed framework integrates fusion and detection across three stages. First, a dual-branch wavelet attention module compares feature maps and identifies areas where one source can provide useful information to the other. It enhances these areas while limiting repeated background responses. Second, a binary fusion Mamba module sends visible and infrared features, as well as their interactions, into state equations as separate inputs. Their origins therefore remain distinguishable during long-range modeling. Third, a three-branch decision head reviews predictions from the visible, infrared, and fused streams. According to scene brightness and the agreement among branches, it adjusts their influence before producing the final boxes and class labels.

The evaluation included repeated trials and compared the framework to representative single-modality and multimodal detectors. They also conducted repeated experiments with different random initializations. Performance was also assessed separately on daytime and nighttime subsets. Removing individual modules reduced detection accuracy, while combining feature selection with dual-input modeling produced the main improvement. The decision stage further improved performance, particularly at night when the infrared branch retained stronger object responses. This result supports the design choice of connecting feature fusion with a later check on prediction consistency.

Cross-dataset experiments employed the same network design for both LLVIP and Aligned FLIR, adjusting only the number of classes and output heads. Since the two datasets differ from DroneVehicle in terms of viewpoint and scene content, they provide a practical test of whether the fusion mechanism depends on a single benchmark. Quantitative comparisons ranked the framework among the top methods, and visual examples revealed fewer missed detections and false positives in low-light, crowded, distant, and cluttered scenes. These observations provide a concrete interpretation of the numerical results without implying that performance will transfer unchanged to every camera or environment.

The study also identifies limitations. The dual-backbone network is too large and slow for strict real-time use on platforms with resource constraints. Results vary by object class; the less-represented Freight category is more difficult than common classes. Additionally, the transfer tests cover three datasets rather than unrestricted operating conditions and use registered visible and infrared image pairs.

Future work will therefore focus on lighter dual-input modules and continuous image sequences, where temporal information and consistency between adjacent frames may stabilize predictions. Broader testing is necessary before using the approach with new sensors, scene types, or computing hardware.