Learning Neuro-symbolic Programs for Language Guided Robot Manipulation. (arXiv:2211.06652v1 [cs.RO])

Given a natural language instruction, and an input and an output scene, our
goal is to train a neuro-symbolic model which can output a manipulation program
that can be executed by the robot on the input scene resulting in the desired
output scene. Prior approaches for this task possess one of the following
limitations: (i) rely on hand-coded symbols for concepts limiting
generalization beyond those seen during training [1] (ii) infer action
sequences from instructions but require dense sub-goal supervision …

arxiv language neuro robot robot manipulation