On the test-time zero-shot generalization of vision-language models: Do we really need prompt learning?

arXiv:2405.02266v1 Announce Type: new
Abstract: The development of large vision-language models, notably CLIP, has catalyzed research into effective adaptation techniques, with a particular focus on soft prompt tuning. Conjointly, test-time augmentation, which utilizes multiple augmented views of a single image to enhance zero-shot generalization, is emerging as a significant area of interest. This has predominantly directed research efforts toward test-time prompt tuning. In contrast, we introduce a robust MeanShift for Test-time Augmentation (MTA), which surpasses prompt-based methods without requiring this …

abstract arxiv augmentation clip cs.cv development focus image language language models multiple prompt prompt learning prompt tuning research test type vision vision-language vision-language models zero-shot