Transducers with Pronunciation-aware Embeddings for Automatic Speech Recognition

arXiv:2404.04295v1 Announce Type: cross
Abstract: This paper proposes Transducers with Pronunciation-aware Embeddings (PET). Unlike conventional Transducers where the decoder embeddings for different tokens are trained independently, the PET model's decoder embedding incorporates shared components for text tokens with the same or similar pronunciations. With experiments conducted in multiple datasets in Mandarin Chinese and Korean, we show that PET models consistently improve speech recognition accuracy compared to conventional Transducers. Our investigation also uncovers a phenomenon that we call error chain reactions. …

abstract arxiv automatic speech recognition chinese components cs.cl cs.lg cs.sd datasets decoder eess.as embedding embeddings multiple paper pet recognition speech speech recognition text the decoder tokens type