i-MAE: Are Latent Representations in Masked Autoencoders Linearly Separable?

arXiv:2210.11470v2 Announce Type: replace-cross
Abstract: Masked image modeling (MIM) has been recognized as a strong self-supervised pre-training approach in the vision domain. However, the mechanism and properties of the learned representations by such a scheme, as well as how to further enhance the representations are so far not well-explored. In this paper, we aim to explore an interactive Masked Autoencoders (i-MAE) framework to enhance the representation capability from two aspects: (1) employing a two-way image reconstruction and a latent feature …

arxiv autoencoders cs.ai cs.cv cs.lg type