MMInA: Benchmarking Multihop Multimodal Internet Agents

arXiv:2404.09992v1 Announce Type: new
Abstract: Autonomous embodied agents live on an Internet of multimedia websites. Can they hop around multimodal websites to complete complex user tasks? Existing benchmarks fail to assess them in a realistic, evolving environment for their embodiment across websites. To answer this question, we present MMInA, a multihop and multimodal benchmark to evaluate the embodied agents for compositional Internet tasks, with several appealing properties: 1) Evolving real-world multimodal websites. Our benchmark uniquely operates on evolving real-world websites, …

agents arxiv benchmarking cs.ai cs.cl cs.cv internet multimodal type