AV-Gaze: A Study on the Effectiveness of Audio Guided Visual Attention Estimation for Non-Profilic Faces. (arXiv:2207.03048v1 [cs.CV])

In challenging real-life conditions such as extreme head-pose, occlusions,
and low-resolution images where the visual information fails to estimate visual
attention/gaze direction, audio signals could provide important and
complementary information. In this paper, we explore if audio-guided coarse
head-pose can further enhance visual attention estimation performance for
non-prolific faces. Since it is difficult to annotate audio signals for
estimating the head-pose of the speaker, we use off-the-shelf state-of-the-art
models to facilitate cross-modal weak-supervision. During the training phase,
the framework learns …

arxiv attention audio av cv study visual attention