Boosting Audio-visual Zero-shot Learning with Large Language Models

arXiv:2311.12268v2 Announce Type: replace
Abstract: Audio-visual zero-shot learning aims to recognize unseen classes based on paired audio-visual sequences. Recent methods mainly focus on learning multi-modal features aligned with class names to enhance the generalization ability to unseen categories. However, these approaches ignore the obscure event concepts in class names and may inevitably introduce complex network structures with difficult training objectives. In this paper, we introduce a straightforward yet efficient framework called KnowleDge-Augmented audio-visual learning (KDA), which aids the model in …

arxiv audio boosting cs.cv language language models large language large language models type visual zero-shot