DreamLIP: Language-Image Pre-training with Long Captions

arXiv:2403.17007v1 Announce Type: new
Abstract: Language-image pre-training largely relies on how precisely and thoroughly a text describes its paired image. In practice, however, the contents of an image can be so rich that well describing them requires lengthy captions (e.g., with 10 sentences), which are usually missing in existing datasets. Consequently, there are currently no clear evidences on whether and how language-image pre-training could benefit from long captions. To figure this out, we first re-caption 30M images with detailed descriptions …

arxiv captions cs.cv image language pre-training training type