Exploiting Inter-sample and Inter-feature Relations in Dataset Distillation

arXiv:2404.00563v1 Announce Type: new
Abstract: Dataset distillation has emerged as a promising approach in deep learning, enabling efficient training with small synthetic datasets derived from larger real ones. Particularly, distribution matching-based distillation methods attract attention thanks to its effectiveness and low computational cost. However, these methods face two primary limitations: the dispersed feature distribution within the same class in synthetic datasets, reducing class discrimination, and an exclusive focus on mean feature consistency, lacking precision and comprehensiveness. To address these challenges, …

arxiv cs.cv dataset distillation feature relations sample type