Improving Speech-to-Speech Translation Through Unlabeled Text. (arXiv:2210.14514v1 [cs.CL])

Direct speech-to-speech translation (S2ST) is among the most challenging
problems in the translation paradigm due to the significant scarcity of S2ST
data. While effort has been made to increase the data size from unlabeled
speech by cascading pretrained speech recognition (ASR), machine translation
(MT) and text-to-speech (TTS) models; unlabeled text has remained relatively
under-utilized to improve S2ST. We propose an effective way to utilize the
massive existing unlabeled text from different languages to create a large
amount of S2ST data …

arxiv speech speech-to-speech translation text translation