Long-Tailed Recognition on Binary Networks by Calibrating A Pre-trained Model

arXiv:2404.00285v1 Announce Type: new
Abstract: Deploying deep models in real-world scenarios entails a number of challenges, including computational efficiency and real-world (e.g., long-tailed) data distributions. We address the combined challenge of learning long-tailed distributions using highly resource-efficient binary neural networks as backbones. Specifically, we propose a calibrate-and-distill framework that uses off-the-shelf pretrained full-precision models trained on balanced datasets to use as teachers for distillation when learning binary networks on long-tailed datasets. To better generalize to various datasets, we further propose …

abstract arxiv binary challenge challenges computational cs.ai cs.cv data efficiency framework networks neural networks pre-trained model recognition type world