A Comprehensive Review of Knowledge Distillation in Computer Vision

arXiv:2404.00936v1 Announce Type: new
Abstract: Deep learning techniques have been demonstrated to surpass preceding cutting-edge machine learning techniques in recent years, with computer vision being one of the most prominent examples. However, deep learning models suffer from significant drawbacks when deployed in resource-constrained environments due to their large model size and high complexity. Knowledge Distillation is one of the prominent solutions to overcome this challenge. This review paper examines the current state of research on knowledge distillation, a technique for …

abstract arxiv computer computer vision cs.cv deep learning deep learning techniques distillation edge environments examples however knowledge machine machine learning machine learning techniques review type vision