ICC: Quantifying Image Caption Concreteness for Multimodal Dataset Curation

arXiv:2403.01306v1 Announce Type: new
Abstract: Web-scale training on paired text-image data is becoming increasingly central to multimodal learning, but is challenged by the highly noisy nature of datasets in the wild. Standard data filtering approaches succeed in removing mismatched text-image pairs, but permit semantically related but highly abstract or subjective text. These approaches lack the fine-grained ability to isolate the most concrete samples that provide the strongest signal for learning in a noisy dataset. In this work, we propose a …

abstract arxiv cs.cv cs.lg curation data dataset datasets filtering image image data multimodal multimodal learning nature scale standard text text-image training type web