[R] MineDreamer: Learning to Follow Instructions via Chain-of-Imagination for Simulated-World Control

**Paper**: [https://arxiv.org/abs/2403.12037](https://arxiv.org/abs/2403.12037)

**Code**: [https://github.com/Zhoues/MineDreamer](https://github.com/Zhoues/MineDreamer)

**Models and Dataset**: [https://huggingface.co/Zhoues](https://huggingface.co/Zhoues)

**Abstract**:

>It is a long-lasting goal to design a generalist-embodied agent that can follow diverse instructions in human-like ways. However, existing approaches often fail to steadily follow instructions due to difficulties in understanding abstract and sequential natural language instructions. To this end, we introduce **MineDreamer**, an open-ended embodied agent built upon the challenging Minecraft simulator with an innovative paradigm that enhances instruction-following ability in low-level control signal generation. Specifically, MineDreamer is developed …

abstract agent design diverse embodied however human human-like language machinelearning minecraft natural natural language paradigm simulator understanding