PEAVS: Perceptual Evaluation of Audio-Visual Synchrony Grounded in Viewers' Opinion Scores

arXiv:2404.07336v1 Announce Type: new
Abstract: Recent advancements in audio-visual generative modeling have been propelled by progress in deep learning and the availability of data-rich benchmarks. However, the growth is not attributed solely to models and benchmarks. Universally accepted evaluation metrics also play an important role in advancing the field. While there are many metrics available to evaluate audio and visual content separately, there is a lack of metrics that offer a quantitative and interpretable measure of audio-visual synchronization for videos …

abstract arxiv audio availability benchmarks cs.cv cs.mm data deep learning eess.as evaluation evaluation metrics generative generative modeling growth however metrics modeling opinion progress role type visual