On the Theory of Cross-Modality Distillation with Contrastive Learning

arXiv:2405.03355v1 Announce Type: new
Abstract: Cross-modality distillation arises as an important topic for data modalities containing limited knowledge such as depth maps and high-quality sketches. Such techniques are of great importance, especially for memory and privacy-restricted scenarios where labeled training data is generally unavailable. To solve the problem, existing label-free methods leverage a few pairwise unlabeled data to distill the knowledge by aligning features or statistics between the source and target modalities. For instance, one typically aims to minimize the …

abstract arxiv cs.cv cs.lg data distillation free importance knowledge maps memory privacy quality sketches solve theory training training data type