Tag2Text: Guiding Vision-Language Model via Image Tagging

arXiv:2303.05657v3 Announce Type: replace
Abstract: This paper presents Tag2Text, a vision language pre-training (VLP) framework, which introduces image tagging into vision-language models to guide the learning of visual-linguistic features. In contrast to prior works which utilize object tags either manually labeled or automatically detected with an off-the-shelf detector with limited performance, our approach explicitly learns an image tagger using tags parsed from image-paired text and thus provides a strong semantic guidance to vision-language models. In this way, Tag2Text can utilize …

arxiv cs.cv image language language model tagging type via vision