MTKD: Multi-Teacher Knowledge Distillation for Image Super-Resolution

arXiv:2404.09571v1 Announce Type: cross
Abstract: Knowledge distillation (KD) has emerged as a promising technique in deep learning, typically employed to enhance a compact student network through learning from their high-performance but more complex teacher variant. When applied in the context of image super-resolution, most KD approaches are modified versions of methods developed for other computer vision tasks, which are based on training strategies with a single teacher and simple loss functions. In this paper, we propose a novel Multi-Teacher Knowledge …

abstract arxiv compact context cs.cv deep learning distillation eess.iv image knowledge network performance resolution through type versions