VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Models. (arXiv:2307.05973v2 [cs.RO] UPDATED)

Large language models (LLMs) are shown to possess a wealth of actionable
knowledge that can be extracted for robot manipulation in the form of reasoning
and planning. Despite the progress, most still rely on pre-defined motion
primitives to carry out the physical interactions with the environment, which
remains a major bottleneck. In this work, we aim to synthesize robot
trajectories, i.e., a dense sequence of 6-DoF end-effector waypoints, for a
large variety of manipulation tasks given an open-set of instructions …

arxiv environment form interactions knowledge language language models large language large language models llms manipulation maps planning progress reasoning robot robotic robot manipulation value wealth