Cross Pseudo-Labeling for Semi-Supervised Audio-Visual Source Localization

arXiv:2403.03095v1 Announce Type: new
Abstract: Audio-Visual Source Localization (AVSL) is the task of identifying specific sounding objects in the scene given audio cues. In our work, we focus on semi-supervised AVSL with pseudo-labeling. To address the issues with vanilla hard pseudo-labels including bias accumulation, noise sensitivity, and instability, we propose a novel method named Cross Pseudo-Labeling (XPL), wherein two models learn from each other with the cross-refine mechanism to avoid bias accumulation. We equip XPL with two effective components. Firstly, …

abstract arxiv audio bias cs.cv cs.mm cs.sd eess.as focus labeling labels localization noise novel objects semi-supervised sensitivity type visual work