DriveVLM: The Convergence of Autonomous Driving and Large Vision-Language Models

arXiv:2402.12289v1 Announce Type: new
Abstract: A primary hurdle of autonomous driving in urban environments is understanding complex and long-tail scenarios, such as challenging road conditions and delicate human behaviors. We introduce DriveVLM, an autonomous driving system leveraging Vision-Language Models (VLMs) for enhanced scene understanding and planning capabilities. DriveVLM integrates a unique combination of chain-of-thought (CoT) modules for scene description, scene analysis, and hierarchical planning. Furthermore, recognizing the limitations of VLMs in spatial reasoning and heavy computational requirements, we propose DriveVLM-Dual, …

abstract arxiv autonomous autonomous driving autonomous driving system capabilities convergence cs.cv driving environments human language language models planning type understanding urban vision vision-language models vlms