Self-supervised 3D anatomy segmentation using self-distilled masked image transformer (SMIT). (arXiv:2205.10342v1 [eess.IV])

Vision transformers, with their ability to more efficiently model long-range
context, have demonstrated impressive accuracy gains in several computer vision
and medical image analysis tasks including segmentation. However, such methods
need large labeled datasets for training, which is hard to obtain for medical
image analysis. Self-supervised learning (SSL) has demonstrated success in
medical image segmentation using convolutional networks. In this work, we
developed a \underline{s}elf-distillation learning with \underline{m}asked
\underline{i}mage modeling method to perform SSL for vision
\underline{t}ransformers (SMIT) applied to …

3d arxiv image segmentation transformer