Textless Direct Speech-to-Speech Translation with Discrete Speech Representation. (arXiv:2211.00115v1 [cs.CL])

Research on speech-to-speech translation (S2ST) has progressed rapidly in
recent years. Many end-to-end systems have been proposed and show advantages
over conventional cascade systems, which are often composed of recognition,
translation and synthesis sub-systems. However, most of the end-to-end systems
still rely on intermediate textual supervision during training, which makes it
infeasible to work for languages without written forms. In this work, we
propose a novel model, Textless Translatotron, which is based on Translatotron
2, for training an end-to-end direct …

arxiv representation speech speech-to-speech translation translation