Label Propagation for Zero-shot Classification with Vision-Language Models

arXiv:2404.04072v1 Announce Type: cross
Abstract: Vision-Language Models (VLMs) have demonstrated impressive performance on zero-shot classification, i.e. classification when provided merely with a list of class names. In this paper, we tackle the case of zero-shot classification in the presence of unlabeled data. We leverage the graph structure of the unlabeled data and introduce ZLaP, a method based on label propagation (LP) that utilizes geodesic distances for classification. We tailor LP to graphs containing both text and image features and further …

arxiv classification cs.cv cs.lg language language models propagation type vision vision-language models zero-shot