Efficient infusion of self-supervised representations in Automatic Speech Recognition

arXiv:2404.12628v1 Announce Type: new
Abstract: Self-supervised learned (SSL) models such as Wav2vec and HuBERT yield state-of-the-art results on speech-related tasks. Given the effectiveness of such models, it is advantageous to use them in conventional ASR systems. While some approaches suggest incorporating these models as a trainable encoder or a learnable frontend, training such systems is extremely slow and requires a lot of computation cycles. In this work, we propose two simple approaches that use (1) framewise addition and (2) cross-attention …

abstract art arxiv asr automatic speech recognition cs.cl encoder frontend recognition results speech speech recognition ssl state systems tasks them type