Visual Clues: Bridging Vision and Language Foundations for Image Paragraph Captioning. (arXiv:2206.01843v2 [cs.CV] UPDATED)

People say, "A picture is worth a thousand words". Then how can we get the
rich information out of the image? We argue that by using visual clues to
bridge large pretrained vision foundation models and language models, we can do
so without any extra cross-modal training. Thanks to the strong zero-shot
capability of foundation models, we start by constructing a rich semantic
representation of the image (e.g., image tags, object attributes / locations,
captions) as a structured textual prompt, …

arxiv captioning image language vision