AVI-Talking: Learning Audio-Visual Instructions for Expressive 3D Talking Face Generation

arXiv:2402.16124v1 Announce Type: new
Abstract: While considerable progress has been made in achieving accurate lip synchronization for 3D speech-driven talking face generation, the task of incorporating expressive facial detail synthesis aligned with the speaker's speaking status remains challenging. Our goal is to directly leverage the inherent style information conveyed by human speech for generating an expressive talking face that aligns with the speaking status. In this paper, we propose AVI-Talking, an Audio-Visual Instruction system for expressive Talking face generation. This …

abstract arxiv audio cs.cv face information progress speaker speaking speech style synchronization synthesis type visual