Multi-scale Bottleneck Transformer for Weakly Supervised Multimodal Violence Detection

arXiv:2405.05130v1 Announce Type: new
Abstract: Weakly supervised multimodal violence detection aims to learn a violence detection model by leveraging multiple modalities such as RGB, optical flow, and audio, while only video-level annotations are available. In the pursuit of effective multimodal violence detection (MVD), information redundancy, modality imbalance, and modality asynchrony are identified as three key challenges. In this work, we propose a new weakly supervised MVD method that explicitly addresses these challenges. Specifically, we introduce a multi-scale bottleneck transformer (MSBT) …

arxiv cs.cv cs.mm detection multimodal scale transformer type violence