DriveDreamer-2: LLM-Enhanced World Models for Diverse Driving Video Generation

arXiv:2403.06845v1 Announce Type: new
Abstract: World models have demonstrated superiority in autonomous driving, particularly in the generation of multi-view driving videos. However, significant challenges still exist in generating customized driving videos. In this paper, we propose DriveDreamer-2, which builds upon the framework of DriveDreamer and incorporates a Large Language Model (LLM) to generate user-defined driving videos. Specifically, an LLM interface is initially incorporated to convert a user's query into agent trajectories. Subsequently, a HDMap, adhering to traffic regulations, is generated …

abstract arxiv autonomous autonomous driving challenges cs.cv diverse driving framework however language language model large language large language model llm paper type video video generation videos view world world models