Towards Efficient foundation models for VHR satellite images

Contexte :
Remote sensing imagery for Earth observation (EO) has emerged as a dynamic research area, enabling precise identification, characterization, and interpretation of objects and materials on the Earth’s surface. The ongoing progress in satellite technology has led to the availability of numerous very-high-resolution (VHR) optical satellites, facilitating daily acquisitions. This enables the creation of highly detailed maps with sub-meter spatial resolution, benefiting various essential EO applications such as urban planning, swift disaster mapping, natural resource management, and wildlife monitoring.

In recent years, deep learning (DL) has found success in various machine learning and computer vision domains, including remote sensing (RS). Despite this, applying DL to real-world scenarios using VHR satellite images for operational purposes faces numerous challenges. The foremost challenge involves the difficulty of annotating domain-specific data, particularly in EO applications that demand expert knowledge. Generating precise and comprehensive labeled datasets for training deep models is a time-consuming and expensive endeavor. For instance, in rapid disaster mapping, acquiring accurate labels is nearly unfeasible due to the infrequent and unique nature of catastrophic events.
Another challenge comes from the significant domain shifts inherent in RS data, arising from diverse sensor characteristics (i.e., spatial resolution and spectral bands) and varied acquisition conditions. Lastly, the exploding size of deep models, with millions (or even billions) of parameters, raises concerns. Not only do they demand substantial computational and storage resources, but they also cause negative environmental issues.

Therefore, designing efficient models while maintaining high accuracy becomes essential in every EO application to reduce energy cost and more importantly, to minimize the environmental impact. These models are expected to be reusable or transferred with low resources. Recent studies have showed that self-supervised pretraining with unlabeled RS images outperforms popular ImageNet-pretrained models in EO downstream tasks, especially when labels are scarce. By leveraging self-supervised learning (SSL) on the abundance of multi-source unlabeled data, foundational models (FMs) have started their era by providing high performance on a wide range of downstream tasks. However, current trends of FMs in EO only focus on creating large vision FMs using substantial multi-source images (RingMo Billion-scale ViT), or large vision-language FMs (RemoteCLIP ). These models demand significant computational resources for training and deployment. Therefore, developing resource-efficient foundation models in the context of EO is imperative to mitigate environmental concerns in the future.

Sujet :
This PhD topic aims to develop efficient foundation models with a focus on EO applications using VHR satellite imagery. The main objectives are the three-folds (please see the details in the attached file or in the below link).
http://www-obelix.irisa.fr/files/2024/02/2024_PhD_IRISA_CNES_Temo.pdf