LongVLM: Efficient Long Video Understanding via Large Language Models

arXiv:2404.03384v1 Announce Type: new
Abstract: Empowered by Large Language Models (LLMs), recent advancements in VideoLLMs have driven progress in various video understanding tasks. These models encode video representations through pooling or query aggregation over a vast number of visual tokens, making computational and memory costs affordable. Despite successfully providing an overall comprehension of video content, existing VideoLLMs still face challenges in achieving detailed understanding in videos due to overlooking local information in long-term videos. To tackle this challenge, we introduce …

arxiv cs.cv language language models large language large language models long video understanding type understanding via video video understanding