Exploring Capabilities of Monolingual Audio Transformers using Large Datasets in Automatic Speech Recognition of Czech. (arXiv:2206.07627v1 [cs.CL])

Web: http://arxiv.org/abs/2206.07627

In this paper, we present our progress in pretraining Czech monolingual audio
transformers from a large dataset containing more than 80 thousand hours of
unlabeled speech, and subsequently fine-tuning the model on automatic speech
recognition tasks using a combination of in-domain data and almost 6 thousand
hours of out-of-domain transcribed speech. We are presenting a large palette of
experiments with various fine-tuning setups evaluated on two public datasets
(CommonVoice and VoxPopuli) and one extremely challenging dataset from the
MALACH project. …

arxiv audio automatic speech recognition datasets large datasets speech speech recognition transformers