ImageInWords: Unlocking Hyper-Detailed Image Descriptions

arXiv:2405.02793v1 Announce Type: new
Abstract: Despite the longstanding adage "an image is worth a thousand words," creating accurate and hyper-detailed image descriptions for training Vision-Language models remains challenging. Current datasets typically have web-scraped descriptions that are short, low-granularity, and often contain details unrelated to the visual content. As a result, models trained on such data generate descriptions replete with missing information, visual inconsistencies, and hallucinations. To address these issues, we introduce ImageInWords (IIW), a carefully designed human-in-the-loop annotation framework for …

abstract arxiv cs.cl cs.cv current data datasets image language language models low training type vision vision-language vision-language models visual web words