HAMMR: HierArchical MultiModal React agents for generic VQA

arXiv:2404.05465v1 Announce Type: cross
Abstract: Combining Large Language Models (LLMs) with external specialized tools (LLMs+tools) is a recent paradigm to solve multimodal tasks such as Visual Question Answering (VQA). While this approach was demonstrated to work well when optimized and evaluated for each individual benchmark, in practice it is crucial for the next generation of real-world AI systems to handle a broad range of multimodal problems. Therefore we pose the VQA problem from a unified perspective and evaluate a single …

abstract agents arxiv benchmark cs.cv cs.lg hierarchical language language models large language large language models llms multimodal paradigm practice question question answering react solve tasks tools type visual vqa work