A Survey on Masked Autoencoder for Self-supervised Learning in Vision and Beyond. (arXiv:2208.00173v1 [cs.CV])

Masked autoencoders are scalable vision learners, as the title of MAE
\cite{he2022masked}, which suggests that self-supervised learning (SSL) in
vision might undertake a similar trajectory as in NLP. Specifically, generative
pretext tasks with the masked prediction (e.g., BERT) have become a de facto
standard SSL practice in NLP. By contrast, early attempts at generative methods
in vision have been buried by their discriminative counterparts (like
contrastive learning); however, the success of mask image modeling has revived
the masking autoencoder (often …

arxiv autoencoder cv learning masked autoencoder self-supervised learning supervised learning survey vision