MILAN: Masked Image Pretraining on Language Assisted Representation. (arXiv:2208.06049v2 [cs.CV] UPDATED)

Self-attention based transformer models have been dominating many computer
vision tasks in the past few years. Their superb model qualities heavily depend
on the excessively large labeled image datasets. In order to reduce the
reliance on large labeled datasets, reconstruction based masked autoencoders
are gaining popularity, which learn high quality transferable representations
from unlabeled images. For the same purpose, recent weakly supervised image
pretraining methods explore language supervision from text captions
accompanying the images. In this work, we propose masked …

arxiv cv image language representation