LLMs Meet Long Video: Advancing Long Video Comprehension with An Interactive Visual Adapter in LLMs

arXiv:2402.13546v1 Announce Type: cross
Abstract: Long video understanding is a significant and ongoing challenge in the intersection of multimedia and artificial intelligence. Employing large language models (LLMs) for comprehending video becomes an emerging and promising method. However, this approach incurs high computational costs due to the extensive array of video tokens, experiences reduced visual clarity as a consequence of token aggregation, and confronts challenges arising from irrelevant visual tokens while answering video-related questions. To alleviate these issues, we present an …

abstract artificial artificial intelligence arxiv challenge computational costs cs.cl cs.cv intelligence interactive intersection language language models large language large language models llms multimedia type understanding video video understanding visual