Li, Haocheng; Zheng, Juepeng; Miao, Shuangxi; Du, Kaiqi; Xiao, Guilong; Zhang, Zhiwei; Zhao, Lingyuan; Fu, Haohuan; Huang, Jianxi
Abstract
Precision agriculture increasingly relies on robust visual understanding of crops and fields. However, agricultural scenes remain challenging due to large appearance variability, background clutter, and fine-grained multiscale structures. Although remote sensing foundation models (RSFMs) have demonstrated strong generalization in generic Earth observation tasks, adapting them effectively to domain-specific agricultural applications is still nontrivial. Existing parameter-efficient fine-tuning (PEFT) strategies often struggle to balance semantic adaptability across diverse crop phenotypes and domain shifts with spatial fidelity in preserving fine-grained structures and boundaries, which limits their effectiveness on dense prediction tasks. To this end, we propose token-driven residual adaptation (TORA), an efficient adaptation paradigm tailored to multiscale agricultural vision. TORA augments frozen RSFMs with hierarchical learnable semantic prototypes to explicitly model domain-specific semantic patterns and to interact with intermediate representations through tokenwise attention. Meanwhile, we design an instance-aware residual path to inject adaptation signals into backbone features, thereby strengthening domain alignment while effectively mitigating the degradation of spatial details. Furthermore, we establish a comprehensive evaluation benchmark spanning three representative tasks-semantic segmentation, object detection, and image classification-across nine datasets. Extensive experiments demonstrate that TORA trains only approximately 1.5%-3.8% of the backbone parameters, substantially fewer than full fine-tuning (FFT), yet achieves performance that is competitive with or superior to strong PEFT baselines and approaches FFT on multiple datasets.
TORA Parameter-Efficient Token-Driven Residual.pdf
