VIMA: Multi-Modal LLM Prompts for Robotics (Stanford, NVIDIA)

A transformer-based robot agent, VIMA, that processes multimodal prompts and outputs motor actions autoregressively. A wide spectrum of robot manipulation tasks can be expressed with multimodal prompts, interleaving textual and visual tokens.

VIMA: General Robot Manipulation with Multimodal Prompts
https://arxiv.org/abs/2210.03094
(all rights with authors)

#ai
#robotics
#multimodal

authors general interleaving llm manipulation multimodal nvidia processes prompts rights robot robotics robot manipulation spectrum stanford textual tokens transformer