您的浏览器版本太低,请使用IE9(或以上)、谷歌、火狐等现代浏览器。360、QQ、搜狗等浏览器请使用极速模式。
学院发表文章

Parameter-efficient multimodal adaptation of vision foundation models for remote sensing semantic segmentation

发布日期:2026-10-04浏览次数:信息来源:土地科学与技术学院

Li, Haocheng;    Zheng, Juepeng;    Miao, Shuangxi;    Lu, Ruibo;    Cai, Guosheng;    Fu, Haohuan;    Huang, Jianxi

Abstract

Multimodal remote sensing semantic segmentation improves scene understanding by integrating complementary information from heterogeneous Earth observation data. Although pretrained Vision Foundation Models (VFMs) provide strong general-purpose representations, adapting them to multimodal tasks typically requires updating large modality-specific encoders, and optimization tends to be dominated by the optical modality, leaving auxiliary cues under-utilized. To address these challenges, we propose MoBaNet, a parameter-efficient multimodal adaptation framework that keeps the VFM backbone frozen and concentrates trainable capacity on three complementary components. First, the Cross-modal Prompt-Injected Adapter (CPIA) generates semantic prompts from paired modalities and injects them into lightweight bottleneck adapters to enable cross-modal interaction under the frozen backbone. Second, the Difference-Guided Gated Fusion Module (DGFM) exploits cross-modal discrepancy to guide channel-wise and spatially adaptive feature fusion. Third, the training-only Modality-Conditional Random Masking (MCRM) strategy locally corrupts one modality at a time and applies hard-pixel auxiliary supervision to modality-specific branches, encouraging more robust use of complementary cues. Extensive experiments on the ISPRS Vaihingen and Potsdam benchmarks show that MoBaNet achieves the best overall performance among the compared methods, while its default DINOv2-ViT-B configuration uses only 5.43% trainable parameters and remains robust under partial modality degradation.


Parameter-efficient multimodal adaptation of vision foundation models for remote sensing semantic segmentation.pdf