SCORE: Self-supervised Correspondence Fine-tuning for Improved Content Representations

arXiv:2403.06260v1 Announce Type: new
Abstract: There is a growing interest in cost-effective self-supervised fine-tuning (SSFT) of self-supervised learning (SSL)-based speech models to obtain task-specific representations. These task-specific representations are used for robust performance on various downstream tasks by fine-tuning on the labelled data. This work presents a cost-effective SSFT method named Self-supervised Correspondence (SCORE) fine-tuning to adapt the SSL speech representations for content-related tasks. The proposed method uses a correspondence training strategy, aiming to learn similar representations from perturbed speech …

abstract arxiv cost cs.cl cs.sd data eess.as fine-tuning performance robust self-supervised learning speech ssl supervised fine-tuning supervised learning tasks type work