SSHR: Leveraging Self-supervised Hierarchical Representations for Multilingual Automatic Speech Recognition

arXiv:2309.16937v2 Announce Type: replace
Abstract: Multilingual automatic speech recognition (ASR) systems have garnered attention for their potential to extend language coverage globally. While self-supervised learning (SSL) models, like MMS, have demonstrated their effectiveness in multilingual ASR, it is worth noting that various layers' representations potentially contain distinct information that has not been fully leveraged. In this study, we propose a novel method that leverages self-supervised hierarchical representations (SSHR) to fine-tune the MMS model. We first analyze the different layers of …

abstract arxiv asr attention automatic speech recognition coverage cs.cl cs.sd eess.as hierarchical information language mms multilingual recognition self-supervised learning speech speech recognition ssl supervised learning systems type while