Dataset Distillation via the Wasserstein Metric

arXiv:2311.18531v2 Announce Type: replace-cross
Abstract: Dataset Distillation (DD) emerges as a powerful strategy to encapsulate the expansive information of large datasets into significantly smaller, synthetic equivalents, thereby preserving model performance with reduced computational overhead. Pursuing this objective, we introduce the Wasserstein distance, a metric grounded in optimal transport theory, to enhance distribution matching in DD. Our approach employs the Wasserstein barycenter to provide a geometrically meaningful method for quantifying distribution differences and capturing the centroid of distribution sets efficiently. By …

abstract arxiv computational cs.ai cs.cv cs.lg dataset datasets distillation distribution information large datasets performance strategy synthetic theory transport type via