Assessing ASR Model Quality on Disordered Speech using BERTScore. (arXiv:2209.10591v1 [eess.AS])

Word Error Rate (WER) is the primary metric used to assess automatic speech
recognition (ASR) model quality. It has been shown that ASR models tend to have
much higher WER on speakers with speech impairments than typical English
speakers. It is hard to determine if models can be be useful at such high error
rates. This study investigates the use of BERTScore, an evaluation metric for
text generation, to provide a more informative measure of ASR model quality and
usefulness. …

arxiv asr quality speech