AstroCLIP: A Cross-Modal Foundation Model for Galaxies

arXiv:2310.03024v2 Announce Type: replace-cross
Abstract: We present AstroCLIP, a single, versatile model that can embed both galaxy images and spectra into a shared, physically meaningful latent space. These embeddings can then be used - without any model fine-tuning - for a variety of downstream tasks including (1) accurate in-modality and cross-modality semantic similarity search, (2) photometric redshift estimation, (3) galaxy property estimation from both images and spectra, and (4) morphology classification. Our approach to implementing AstroCLIP consists of two parts. …

abstract arxiv astro-ph.im cs.ai cs.lg embed embeddings fine-tuning foundation foundation model galaxy images modal model fine-tuning replace semantic space tasks tuning type