Vision-Language Interpreter for Robot Task Planning

arXiv:2311.00967v2 Announce Type: replace-cross
Abstract: Large language models (LLMs) are accelerating the development of language-guided robot planners. Meanwhile, symbolic planners offer the advantage of interpretability. This paper proposes a new task that bridges these two trends, namely, multimodal planning problem specification. The aim is to generate a problem description (PD), a machine-readable file used by the planners to find a plan. By generating PDs from language instruction and scene observation, we can drive symbolic planners in a language-guided framework. We …

arxiv cs.ai cs.cl cs.ro interpreter language planning robot type vision