MineDreamer: Learning to Follow Instructions via Chain-of-Imagination for Simulated-World Control

arXiv:2403.12037v1 Announce Type: new
Abstract: It is a long-lasting goal to design a generalist-embodied agent that can follow diverse instructions in human-like ways. However, existing approaches often fail to steadily follow instructions due to difficulties in understanding abstract and sequential natural language instructions. To this end, we introduce MineDreamer, an open-ended embodied agent built upon the challenging Minecraft simulator with an innovative paradigm that enhances instruction-following ability in low-level control signal generation. Specifically, MineDreamer is developed on top of recent …

abstract agent arxiv control cs.cv design diverse embodied however human human-like imagination language natural natural language type understanding via world