The Audio-Visual Conversational Graph: From an Egocentric-Exocentric Perspective

arXiv:2312.12870v2 Announce Type: replace
Abstract: In recent years, the thriving development of research related to egocentric videos has provided a unique perspective for the study of conversational interactions, where both visual and audio signals play a crucial role. While most prior work focus on learning about behaviors that directly involve the camera wearer, we introduce the Ego-Exocentric Conversational Graph Prediction problem, marking the first attempt to infer exocentric conversational interactions from egocentric videos. We propose a unified multi-modal framework -- …

arxiv audio conversational cs.cv graph perspective type visual