Parameter-Efficient Adaptation of Geospatial Foundation Models through Embedding Deflection

As large-scale heterogeneous data sets become increasingly available, adapting Foundation Models at low cost has become a key issue.Seminal works in natural language processing, e.g. Low-Rank Adaptation (LoRA), leverage the low "intrinsic rank" of parameter updates during adaptation. In this paper, we argue that stronger inductive biases on the data and on the models can improve the adaptation of Foundation Models pretrained on RGB satellite images to other sources of satellite data. The pretrained parameters of Geospatial Foundation Models (GFMs) indeed provide a strong prior on the spatial dimension of multispectral images. For this reason, we introduce DEFLECT (Deflecting Embeddings for Finetuning Latent representations for Earth and Climate Tasks), a novel strategy for adapting GFMs to multispectral satellite imagery with very few additional parameters. DEFLECT improves the representation capabilities of the extracted features, particularly enhancing spectral information, which is essential for geoscience and environmental-related tasks. We demonstrate the effectiveness of our method across three different GFMs and five diverse datasets, ranging from forest monitoring to marine environment segmentation. Compared to competing methods, DEFLECT achieves on-par or higher accuracy with 5-10x fewer parameters for classification and segmentation tasks. The code will be made publicly available.