Downstream Task Guided Masking Learning in Masked Autoencoders Using Multi-Level Optimization

arXiv:2402.18128v1 Announce Type: cross
Abstract: Masked Autoencoder (MAE) is a notable method for self-supervised pretraining in visual representation learning. It operates by randomly masking image patches and reconstructing these masked patches using the unmasked ones. A key limitation of MAE lies in its disregard for the varying informativeness of different patches, as it uniformly selects patches to mask. To overcome this, some approaches propose masking based on patch informativeness. However, these methods often do not consider the specific requirements of …

arxiv autoencoders cs.cv cs.lg masking optimization type