Practical Dataset Distillation Based on Deep Support Vectors

arXiv:2405.00348v1 Announce Type: new
Abstract: Conventional dataset distillation requires significant computational resources and assumes access to the entire dataset, an assumption impractical as it presumes all data resides on a central server. In this paper, we focus on dataset distillation in practical scenarios with access to only a fraction of the entire dataset. We introduce a novel distillation method that augments the conventional process by incorporating general model knowledge via the addition of Deep KKT (DKKT) loss. In practical settings, …

abstract access arxiv computational cs.lg data dataset distillation focus paper practical resources server support type vectors