Data-Efficient Contrastive Language-Image Pretraining: Prioritizing Data Quality over Quantity

arXiv:2403.12267v1 Announce Type: cross
Abstract: Contrastive Language-Image Pre-training (CLIP) on large-scale image-caption datasets learns representations that can achieve remarkable zero-shot generalization. However, such models require a massive amount of pre-training data. Improving the quality of the pre-training data has been shown to be much more effective in improving CLIP's performance than increasing its volume. Nevertheless, finding small subsets of training data that provably generalize the best has remained an open question. In this work, we propose the first theoretically rigorous …

arxiv cs.cv cs.lg data data quality image language pretraining quality type