Does Whisper understand Swiss German? An automatic, qualitative, and human evaluation

arXiv:2404.19310v1 Announce Type: new
Abstract: Whisper is a state-of-the-art automatic speech recognition (ASR) model (Radford et al., 2022). Although Swiss German dialects are allegedly not part of Whisper's training data, preliminary experiments showed that Whisper can transcribe Swiss German quite well, with the output being a speech translation into Standard German. To gain a better understanding of Whisper's performance on Swiss German, we systematically evaluate it using automatic, qualitative, and human evaluation. We test its performance on three existing test …

abstract art arxiv asr automatic speech recognition cs.cl data evaluation german human part recognition speech speech recognition state training training data transcribe translation type whisper