PLLaVA : Parameter-free LLaVA Extension from Images to Videos for Video Dense Captioning

arXiv:2404.16994v1 Announce Type: new
Abstract: Vision-language pre-training has significantly elevated performance across a wide range of image-language applications. Yet, the pre-training process for video-related tasks demands exceptionally large computational and data resources, which hinders the progress of video-language models. This paper investigates a straightforward, highly efficient, and resource-light approach to adapting an existing image-language pre-trained model for dense video understanding. Our preliminary experiments reveal that directly fine-tuning pre-trained image-language models with multiple frames as inputs on video datasets leads to …

arxiv captioning cs.cv extension free images llava type video videos