Copilot4D: Learning Unsupervised World Models for Autonomous Driving via Discrete Diffusion

arXiv:2311.01017v4 Announce Type: replace-cross
Abstract: Learning world models can teach an agent how the world works in an unsupervised manner. Even though it can be viewed as a special case of sequence modeling, progress for scaling world models on robotic applications such as autonomous driving has been somewhat less rapid than scaling language models with Generative Pre-trained Transformers (GPT). We identify two reasons as major bottlenecks: dealing with complex and unstructured observation space, and having a scalable generative model. Consequently, …

abstract agent applications arxiv autonomous autonomous driving case cs.ai cs.cv cs.lg cs.ro diffusion driving modeling progress robotic scaling type unsupervised via world world models