End-to-End Video-To-Speech Synthesis using Generative Adversarial Networks. (arXiv:2104.13332v3 [cs.LG] UPDATED)

Video-to-speech is the process of reconstructing the audio speech from a
video of a spoken utterance. Previous approaches to this task have relied on a
two-step process where an intermediate representation is inferred from the
video, and is then decoded into waveform audio using a vocoder or a waveform
reconstruction algorithm. In this work, we propose a new end-to-end
video-to-speech model based on Generative Adversarial Networks (GANs) which
translates spoken video to waveform end-to-end without using any intermediate
representation or …

arxiv generative adversarial networks lg networks speech video