CtrlFormer: Learning Transferable State Representation for Visual Control via Transformer. (arXiv:2206.08883v1 [cs.CV])

Transformer has achieved great successes in learning vision and language
representation, which is general across various downstream tasks. In visual
control, learning transferable state representation that can transfer between
different control tasks is important to reduce the training sample size.
However, porting Transformer to sample-efficient visual control remains a
challenging and unsolved problem. To this end, we propose a novel Control
Transformer (CtrlFormer), possessing many appealing benefits that prior arts do
not have. Firstly, CtrlFormer jointly learns self-attention mechanisms between …

arxiv cv learning representation state transformer