Leveraging Speech for Gesture Detection in Multimodal Communication

arXiv:2404.14952v1 Announce Type: new
Abstract: Gestures are inherent to human interaction and often complement speech in face-to-face communication, forming a multimodal communication system. An important task in gesture analysis is detecting a gesture's beginning and end. Research on automatic gesture detection has primarily focused on visual and kinematic information to detect a limited set of isolated or silent gestures with low variability, neglecting the integration of speech and vision signals to detect gestures that co-occur with speech. This work addresses …

abstract analysis arxiv communication cs.ai cs.cv detection face gestures human information multimodal research speech type visual