DriveGPT4: Interpretable End-to-end Autonomous Driving via Large Language Model

Multimodal large language models (MLLMs) have emerged as a prominent area of interest within the research community, given their proficiency in handling and reasoning with non-textual data, including images and videos. This study seeks to extend the application of MLLMs to the realm of autonomous driving by introducing DriveGPT4, a novel interpretable end-to-end autonomous driving system based on LLMs. Capable of processing multi-frame video inputs and textual queries, DriveGPT4 facilitates the interpretation of vehicle actions, offers pertinent reasoning, and effectively …

application autonomous autonomous driving community cs.cv cs.ro data driving images language language model language models large language large language model large language models mllms multimodal novel reasoning research research community study textual via videos