Semantic Gesticulator: Semantics-Aware Co-Speech Gesture Synthesis

arXiv:2405.09814v1 Announce Type: cross
Abstract: In this work, we present Semantic Gesticulator, a novel framework designed to synthesize realistic gestures accompanying speech with strong semantic correspondence. Semantically meaningful gestures are crucial for effective non-verbal communication, but such gestures often fall within the long tail of the distribution of natural human motion. The sparsity of these movements makes it challenging for deep learning-based systems, trained on moderately sized datasets, to capture the relationship between the movements and the corresponding speech semantics. …

abstract arxiv communication cs.cv cs.gr cs.sd distribution eess.as framework gestures human natural novel semantic semantics sparsity speech synthesis type verbal work