Dataset Distillation using Parameter Pruning. (arXiv:2209.14609v2 [cs.CV] UPDATED)

The acquisition of advanced models relies on large datasets in many fields,
which makes storing datasets and training models expensive. As a solution,
dataset distillation can synthesize a small dataset such that models trained on
it achieve high performance on par with the original large dataset. The
recently proposed dataset distillation method by matching network parameters
has been proved effective for several datasets. However, a few parameters in
the distillation process are difficult to match, which harms the distillation
performance. …

arxiv dataset distillation pruning