Lightweight, Pre-trained Transformers for Remote Sensing Timeseries

Machine learning methods for satellite data have a range of societally relevant applications, but labels used to train models can be difficult or impossible to acquire. Self-supervision is a natural solution in settings with limited labeled data, but current self-supervised models for satellite data fail to take advantage of the characteristics of that data, including the temporal dimension (which is critical for many applications, such as monitoring crop growth) and availability of data from many complementary sensors (which can significantly …

applications cs.ai cs.cv current data labels machine machine learning natural satellite sensing solution supervision timeseries train transformers