Promoting CNNs with Cross-Architecture Knowledge Distillation for Efficient Monocular Depth Estimation

arXiv:2404.16386v1 Announce Type: new
Abstract: Recently, the performance of monocular depth estimation (MDE) has been significantly boosted with the integration of transformer models. However, the transformer models are usually computationally-expensive, and their effectiveness in light-weight models are limited compared to convolutions. This limitation hinders their deployment on resource-limited devices. In this paper, we propose a cross-architecture knowledge distillation method for MDE, dubbed DisDepth, to enhance efficient CNN models with the supervision of state-of-the-art transformer models. Concretely, we first build a …

abstract architecture arxiv cnns cs.cv deployment devices distillation however integration knowledge light mde performance transformer transformer models type