G4G:A Generic Framework for High Fidelity Talking Face Generation with Fine-grained Intra-modal Alignment

arXiv:2402.18122v1 Announce Type: new
Abstract: Despite numerous completed studies, achieving high fidelity talking face generation with highly synchronized lip movements corresponding to arbitrary audio remains a significant challenge in the field. The shortcomings of published studies continue to confuse many researchers. This paper introduces G4G, a generic framework for high fidelity talking face generation with fine-grained intra-modal alignment. G4G can reenact the high fidelity of original video while producing highly synchronized lip movements regardless of given audio tones or volumes. …

abstract alignment arxiv audio challenge cs.cv cs.mm face fidelity fine-grained framework modal movements paper researchers studies type