Perceiver-Actor: A Multi-Task Transformer for Robotic Manipulation. (arXiv:2209.05451v2 [cs.RO] UPDATED)

Transformers have revolutionized vision and natural language processing with
their ability to scale with large datasets. But in robotic manipulation, data
is both limited and expensive. Can manipulation still benefit from Transformers
with the right problem formulation? We investigate this question with PerAct, a
language-conditioned behavior-cloning agent for multi-task 6-DoF manipulation.
PerAct encodes language goals and RGB-D voxel observations with a Perceiver
Transformer, and outputs discretized actions by ``detecting the next best voxel
action''. Unlike frameworks that operate on 2D …

arxiv perceiver transformer