Textless Low-Resource Speech-to-Speech Translation With Unit Language Models

arXiv:2305.15405v2 Announce Type: replace
Abstract: Existing speech-to-speech translation models fall into two camps: textless models trained with hundreds of hours of parallel speech data or unsupervised models that leverage text as an intermediate step. Both approaches limit building speech-to-speech translation models for a wide range of languages, as they exclude languages that are primarily spoken and language pairs that lack large-scale parallel speech data. We present a new framework for training textless low-resource speech-to-speech translation (S2ST) systems that only need …

abstract arxiv building cs.cl data eess.as intermediate language language models languages low speech speech-to-speech translation text translation type unsupervised