Any-Shift Prompting for Generalization over Distributions

arXiv:2402.10099v1 Announce Type: new
Abstract: Image-language models with prompt learning have shown remarkable advances in numerous downstream vision tasks. Nevertheless, conventional prompt learning methods overfit their training distribution and lose the generalization ability on test distributions. To improve generalization across various distribution shifts, we propose any-shift prompting: a general probabilistic inference framework that considers the relationship between training and test distributions during prompt learning. We explicitly connect training and test distributions in the latent space by constructing training and test …

abstract advances arxiv cs.cv distribution framework general image inference language language models prompt prompting prompt learning shift tasks test training type vision