EfficientTrain: Exploring Generalized Curriculum Learning for Training Visual Backbones. (arXiv:2211.09703v1 [cs.CV])

The superior performance of modern deep networks usually comes at the price
of a costly training procedure. In this paper, we present a novel curriculum
learning approach for the efficient training of visual backbones (e.g., vision
Transformers). The proposed method is inspired by the phenomenon that deep
networks mainly learn to recognize some 'easier-to-learn' discriminative
patterns within each example at earlier stages of training, e.g., the
lower-frequency components of images and the original information before data
augmentation. Driven by this …

arxiv curriculum curriculum learning training