ATOM: Attention Mixer for Efficient Dataset Distillation

arXiv:2405.01373v1 Announce Type: new
Abstract: Recent works in dataset distillation seek to minimize training expenses by generating a condensed synthetic dataset that encapsulates the information present in a larger real dataset. These approaches ultimately aim to attain test accuracy levels akin to those achieved by models trained on the entirety of the original dataset. Previous studies in feature and distribution matching have achieved significant results without incurring the costs of bi-level optimization in the distillation process. Despite their convincing efficiency, …

abstract accuracy aim arxiv atom attention cs.cv dataset distillation information seek synthetic test the information training type