Distinctive Image Captioning via CLIP Guided Group Optimization. (arXiv:2208.04254v5 [cs.CV] UPDATED)

Image captioning models are usually trained according to human annotated
ground-truth captions, which could generate accurate but generic captions. In
this paper, we focus on generating distinctive captions that can distinguish
the target image from other similar images. To evaluate the distinctiveness of
captions, we introduce a series of metrics that use large-scale vision-language
pre-training model CLIP to quantify the distinctiveness. To further improve the
distinctiveness of captioning models, we propose a simple and effective
training strategy that trains the …

arxiv captioning clip image optimization