iRAG: An Incremental Retrieval Augmented Generation System for Videos

arXiv:2404.12309v1 Announce Type: cross
Abstract: Retrieval augmented generation (RAG) systems combine the strengths of language generation and information retrieval to power many real-world applications like chatbots. Use of RAG for combined understanding of multimodal data such as text, images and videos is appealing but two critical limitations exist: one-time, upfront capture of all content in large multimodal data as text descriptions entails high processing times, and not all information in the rich multimodal data is typically in the text descriptions. …

abstract applications arxiv chatbots cs.cv cs.ir cs.lg data images incremental information language language generation limitations multimodal multimodal data power rag retrieval retrieval augmented generation systems text type understanding videos world