MAGE: MAsked Generative Encoder to Unify Representation Learning and Image Synthesis. (arXiv:2211.09117v1 [cs.CV])

Generative modeling and representation learning are two key tasks in computer
vision. However, these models are typically trained independently, which
ignores the potential for each task to help the other, and leads to training
and model maintenance overheads. In this work, we propose MAsked Generative
Encoder (MAGE), the first framework to unify SOTA image generation and
self-supervised representation learning. Our key insight is that using variable
masking ratios in masked image modeling pre-training can allow generative
training (very high masking …

arxiv encoder image mage representation representation learning