T-MARS: Improving Visual Representations by Circumventing Text Feature Learning

arXiv:2307.03132v2 Announce Type: replace-cross
Abstract: Large web-sourced multimodal datasets have powered a slew of new methods for learning general-purpose visual representations, advancing the state of the art in computer vision and revolutionizing zero- and few-shot recognition. One crucial decision facing practitioners is how, if at all, to curate these ever-larger datasets. For example, the creators of the LAION-5B dataset chose to retain only image-caption pairs whose CLIP similarity score exceeded a designated threshold. In this paper, we propose a new …

arxiv cs.cl cs.cv cs.lg feature mars text type visual