LEGENT: Open Platform for Embodied Agents

arXiv:2404.18243v1 Announce Type: new
Abstract: Despite advancements in Large Language Models (LLMs) and Large Multimodal Models (LMMs), their integration into language-grounded, human-like embodied agents remains incomplete, hindering complex real-life task performance in physical environments. Existing integrations often feature limited open sourcing, challenging collective progress in this field. We introduce LEGENT, an open, scalable platform for developing embodied agents using LLMs and LMMs. LEGENT offers a dual approach: a rich, interactive 3D environment with communicable and actionable agents, paired with a …

abstract agents arxiv collective cs.cl embodied environments feature human human-like integration integrations language language models large language large language models large multimodal models life llms lmms multimodal multimodal models performance platform progress scalable type