Adapting Dual-encoder Vision-language Models for Paraphrased Retrieval

arXiv:2405.03190v1 Announce Type: new
Abstract: In the recent years, the dual-encoder vision-language models (\eg CLIP) have achieved remarkable text-to-image retrieval performance. However, we discover that these models usually results in very different retrievals for a pair of paraphrased queries. Such behavior might render the retrieval system less predictable and lead to user frustration. In this work, we consider the task of paraphrased text-to-image retrieval where a model aims to return similar results given a pair of paraphrased queries. To start …

abstract arxiv behavior clip cs.cv encoder however image language language models performance queries render results retrieval text text-to-image type vision vision-language vision-language models