Visual and audio scene classification for detecting discrepancies in video: a baseline method and experimental protocol

arXiv:2405.00384v1 Announce Type: new
Abstract: This paper presents a baseline approach and an experimental protocol for a specific content verification problem: detecting discrepancies between the audio and video modalities in multimedia content. We first design and optimize an audio-visual scene classifier, to compare with existing classification baselines that use both modalities. Then, by applying this classifier separately to the audio and the visual modality, we can detect scene-class inconsistencies between them. To facilitate further research and provide a common evaluation …

abstract arxiv audio classification classifier cs.cv cs.mm cs.sd design eess.as experimental multimedia paper protocol type verification video visual