Towards Representative Subset Selection for Self-Supervised Speech Recognition. (arXiv:2203.09829v2 [cs.LG] UPDATED)

Self-supervised speech recognition models require considerable labeled
training data for learning high-fidelity representations for Automatic Speech
Recognition (ASR) which is computationally demanding and time-consuming,
thereby hindering the usage of these models in resource-constrained
environments. We consider the task of identifying an optimal subset of data to
train self-supervised speech models for ASR. We make a surprising observation
that the dataset pruning strategies used in vision tasks for sampling the most
informative examples do not perform better than random subset selection …

arxiv lg speech speech recognition