Robust Active Speaker Detection in Noisy Environments

arXiv:2403.19002v1 Announce Type: cross
Abstract: This paper addresses the issue of active speaker detection (ASD) in noisy environments and formulates a robust active speaker detection (rASD) problem. Existing ASD approaches leverage both audio and visual modalities, but non-speech sounds in the surrounding environment can negatively impact performance. To overcome this, we propose a novel framework that utilizes audio-visual speech separation as guidance to learn noise-free audio features. These features are then utilized in an ASD model, and both tasks are …

abstract arxiv audio cs.cv cs.mm cs.sd detection eess.as environment environments impact issue paper performance robust speaker speech type visual