Do We Really Need a Complex Agent System? Distill Embodied Agent into a Single Model

arXiv:2404.04619v1 Announce Type: cross
Abstract: With the power of large language models (LLMs), open-ended embodied agents can flexibly understand human instructions, generate interpretable guidance strategies, and output executable actions. Nowadays, Multi-modal Language Models~(MLMs) integrate multi-modal signals into LLMs, further bringing richer perception to entity agents and allowing embodied agents to perceive world-understanding tasks more delicately. However, existing works: 1) operate independently by agents, each containing multiple LLMs, from perception to action, resulting in gaps between complex tasks and execution; 2) …

abstract agent agents arxiv cs.ai cs.cv embodied generate guidance human language language models large language large language models llms modal multi-modal perception power strategies type