MeaCap: Memory-Augmented Zero-shot Image Captioning

arXiv:2403.03715v1 Announce Type: new
Abstract: Zero-shot image captioning (IC) without well-paired image-text data can be divided into two categories, training-free and text-only-training. Generally, these two types of methods realize zero-shot IC by integrating pretrained vision-language models like CLIP for image-text similarity evaluation and a pre-trained language model (LM) for caption generation. The main difference between them is whether using a textual corpus to train the LM. Though achieving attractive performance w.r.t. some metrics, existing methods often exhibit some common drawbacks. …

arxiv captioning cs.cv image memory type zero-shot