The Use of Multimodal Large Language Models to Detect Objects from Thermal Images: Transportation Applications

arXiv:2406.13898v1 Announce Type: cross
Abstract: The integration of thermal imaging data with Multimodal Large Language Models (MLLMs) constitutes an exciting opportunity for improving the safety and functionality of autonomous driving systems and many Intelligent Transportation Systems (ITS) applications. This study investigates whether MLLMs can understand complex images from RGB and thermal cameras and detect objects directly. Our goals were to 1) assess the ability of the MLLM to learn from information from various sets, 2) detect objects and identify elements …

abstract applications arxiv autonomous autonomous driving autonomous driving systems cs.cl cs.cv cs.cy data driving images imaging improving integration intelligent intelligent transportation language language models large language large language models mllms multimodal objects safety study systems transportation type