Knowledge Distillation with Multi-granularity Mixture of Priors for Image Super-Resolution

arXiv:2404.02573v1 Announce Type: new
Abstract: Knowledge distillation (KD) is a promising yet challenging model compression technique that transfers rich learning representations from a well-performing but cumbersome teacher model to a compact student model. Previous methods for image super-resolution (SR) mostly compare the feature maps directly or after standardizing the dimensions with basic algebraic operations (e.g. average, dot-product). However, the intrinsic semantic differences among feature maps are overlooked, which are caused by the disparate expressive capacity between the networks. This work …

abstract arxiv compact compression cs.cv dimensions distillation feature image knowledge maps resolution type