Instruction-driven history-aware policies for robotic manipulations. (arXiv:2209.04899v2 [cs.RO] UPDATED)

In human environments, robots are expected to accomplish a variety of
manipulation tasks given simple natural language instructions. Yet, robotic
manipulation is extremely challenging as it requires fine-grained motor
control, long-term memory as well as generalization to previously unseen tasks
and environments. To address these challenges, we propose a unified
transformer-based approach that takes into account multiple inputs. In
particular, our transformer architecture integrates (i) natural language
instructions and (ii) multi-view scene observations while (iii) keeping track
of the full …

arxiv history