Masked Image Modeling with Denoising Contrast. (arXiv:2205.09616v1 [cs.CV])

Since the development of self-supervised visual representation learning from
contrastive learning to masked image modeling, there is no significant
difference in essence, that is, how to design proper pretext tasks for vision
dictionary look-up. Masked image modeling recently dominates this line of
research with state-of-the-art performance on vision Transformers, where the
core is to enhance the patch-level visual context capturing of the network via
denoising auto-encoding mechanism. Rather than tailoring image tokenizers with
extra training stages as in previous works, …

arxiv cv denoising image modeling