OmniSat: Self-Supervised Modality Fusion for Earth Observation

arXiv:2404.08351v1 Announce Type: new
Abstract: The field of Earth Observations (EO) offers a wealth of data from diverse sensors, presenting a great opportunity for advancing self-supervised multimodal learning. However, current multimodal EO datasets and models focus on a single data type, either mono-date images or time series, which limits their expressivity. We introduce OmniSat, a novel architecture that exploits the spatial alignment between multiple EO modalities to learn expressive multimodal representations without labels. To demonstrate the advantages of combining modalities …

abstract arxiv cs.cv current data datasets diverse earth earth observation focus fusion however images multimodal multimodal learning observation presenting sensors series time series type wealth