MixSKD: Self-Knowledge Distillation from Mixup for Image Recognition. (arXiv:2208.05768v1 [cs.CV])

Unlike the conventional Knowledge Distillation (KD), Self-KD allows a network
to learn knowledge from itself without any guidance from extra networks. This
paper proposes to perform Self-KD from image Mixture (MixSKD), which integrates
these two techniques into a unified framework. MixSKD mutually distills feature
maps and probability distributions between the random pair of original images
and their mixup images in a meaningful way. Therefore, it guides the network to
learn cross-image knowledge by modelling supervisory signals from mixup images.
Moreover, …

arxiv cv distillation image image recognition knowledge