LiT: Zero-Shot Transfer with Locked-image text Tuning. (arXiv:2111.07991v3 [cs.CV] UPDATED)

Web: http://arxiv.org/abs/2111.07991

This paper presents contrastive-tuning, a simple method employing contrastive
training to align image and text models while still taking advantage of their
pre-training. In our empirical study we find that locked pre-trained image
models with unlocked text models work best. We call this instance of
contrastive-tuning "Locked-image Tuning" (LiT), which just teaches a text model
to read out good representations from a pre-trained image model for new tasks.
A LiT model gains the capability of zero-shot transfer to new vision …

arxiv cv image text transfer