UniAV: Unified Audio-Visual Perception for Multi-Task Video Localization

arXiv:2404.03179v1 Announce Type: new
Abstract: Video localization tasks aim to temporally locate specific instances in videos, including temporal action localization (TAL), sound event detection (SED) and audio-visual event localization (AVEL). Existing methods over-specialize on each task, overlooking the fact that these instances often occur in the same video to form the complete video content. In this work, we present UniAV, a Unified Audio-Visual perception network, to achieve joint learning of TAL, SED and AVEL tasks for the first time. UniAV …

abstract aim arxiv audio cs.cv cs.mm cs.sd detection eess.as event form instances localization perception sound tasks temporal type video videos visual