Transfer Learning from Whisper for Microscopic Intelligibility Prediction

arXiv:2404.01737v1 Announce Type: cross
Abstract: Macroscopic intelligibility models predict the expected human word-error-rate for a given speech-in-noise stimulus. In contrast, microscopic intelligibility models aim to make fine-grained predictions about listeners' perception, e.g. predicting phonetic or lexical responses. State-of-the-art macroscopic models use transfer learning from large scale deep learning models for speech processing, whereas such methods have rarely been used for microscopic modeling. In this paper, we study the use of transfer learning from Whisper, a state-of-the-art deep learning model for …

abstract aim art arxiv contrast cs.cl cs.sd deep learning eess.as error fine-grained human noise perception prediction predictions processing rate responses scale speech speech processing state stimulus transfer transfer learning type whisper word