CAE v2: Context Autoencoder with CLIP Target. (arXiv:2211.09799v1 [cs.CV])

Masked image modeling (MIM) learns visual representation by masking and
reconstructing image patches. Applying the reconstruction supervision on the
CLIP representation has been proven effective for MIM. However, it is still
under-explored how CLIP supervision in MIM influences performance. To
investigate strategies for refining the CLIP-targeted MIM, we study two
critical elements in MIM, i.e., the supervision position and the mask ratio,
and reveal two interesting perspectives, relying on our developed simple
pipeline, context autodecoder with CLIP target (CAE v2). …

arxiv autoencoder cae clip context