DailyMAE: Towards Pretraining Masked Autoencoders in One Day

arXiv:2404.00509v1 Announce Type: new
Abstract: Recently, masked image modeling (MIM), an important self-supervised learning (SSL) method, has drawn attention for its effectiveness in learning data representation from unlabeled data. Numerous studies underscore the advantages of MIM, highlighting how models pretrained on extensive datasets can enhance the performance of downstream tasks. However, the high computational demands of pretraining pose significant challenges, particularly within academic environments, thereby impeding the SSL research progress. In this study, we propose efficient training recipes for MIM …

arxiv autoencoders cs.cv cs.lg pretraining type