Co-Speech Gesture Detection through Multi-Phase Sequence Labeling

arXiv:2308.10680v2 Announce Type: replace
Abstract: Gestures are integral components of face-to-face communication. They unfold over time, often following predictable movement phases of preparation, stroke, and retraction. Yet, the prevalent approach to automatic gesture detection treats the problem as binary classification, classifying a segment as either containing a gesture or not, thus failing to capture its inherently sequential and contextual nature. To address this, we introduce a novel framework that reframes the task as a multi-phase sequence labeling problem rather than …

abstract arxiv binary classification communication components cs.cv detection face gestures integral labeling segment speech stroke through type