V2Meow: Meowing to the Visual Beat via Video-to-Music Generation

arXiv:2305.06594v2 Announce Type: replace-cross
Abstract: Video-to-music generation demands both a temporally localized high-quality listening experience and globally aligned video-acoustic signatures. While recent music generation models excel at the former through advanced audio codecs, the exploration of video-acoustic signatures has been confined to specific visual scenarios. In contrast, our research confronts the challenge of learning globally aligned signatures between video and music directly from paired music and videos, without explicitly modeling domain-specific rhythmic or semantic relationships. We propose V2Meow, a video-to-music …

abstract advanced arxiv audio contrast cs.cv cs.lg cs.mm cs.sd eess.as excel experience exploration music music generation quality research through type via video visual