MA-LMM: Memory-Augmented Large Multimodal Model for Long-Term Video Understanding

arXiv:2404.05726v1 Announce Type: new
Abstract: With the success of large language models (LLMs), integrating the vision model into LLMs to build vision-language foundation models has gained much more interest recently. However, existing LLM-based large multimodal models (e.g., Video-LLaMA, VideoChat) can only take in a limited number of frames for short video understanding. In this study, we mainly focus on designing an efficient and effective model for long-term video understanding. Instead of trying to process more frames simultaneously like most existing …

arxiv cs.cv lmm long-term memory multimodal multimodal model type understanding video video understanding