Surfer: Progressive Reasoning with World Models for Robotic Manipulation

arXiv:2306.11335v4 Announce Type: replace-cross
Abstract: Considering how to make the model accurately understand and follow natural language instructions and perform actions consistent with world knowledge is a key challenge in robot manipulation. This mainly includes human fuzzy instruction reasoning and the following of physical knowledge. Therefore, the embodied intelligence agent must have the ability to model world knowledge from training data. However, most existing vision and language robot manipulation methods mainly operate in less realistic simulator and language settings and …

abstract agent arxiv challenge consistent cs.ai cs.cv cs.lg cs.ro embodied embodied intelligence human intelligence key knowledge language manipulation natural natural language reasoning robot robotic robotic manipulation robot manipulation type world world models