MiM: Mask in Mask Self-Supervised Pre-Training for 3D Medical Image Analysis

arXiv:2404.15580v1 Announce Type: new
Abstract: The Vision Transformer (ViT) has demonstrated remarkable performance in Self-Supervised Learning (SSL) for 3D medical image analysis. Mask AutoEncoder (MAE) for feature pre-training can further unleash the potential of ViT on various medical vision tasks. However, due to large spatial sizes with much higher dimensions of 3D medical images, the lack of hierarchical design for MAE may hinder the performance of downstream tasks. In this paper, we propose a novel \textit{Mask in Mask (MiM)} pre-training …

abstract analysis arxiv autoencoder cs.cv feature however image medical performance pre-training self-supervised learning spatial ssl supervised learning tasks training transformer type vision vit