Driver Activity Classification Using Generalizable Representations from Vision-Language Models

arXiv:2404.14906v1 Announce Type: cross
Abstract: Driver activity classification is crucial for ensuring road safety, with applications ranging from driver assistance systems to autonomous vehicle control transitions. In this paper, we present a novel approach leveraging generalizable representations from vision-language models for driver activity classification. Our method employs a Semantic Representation Late Fusion Neural Network (SRLF-Net) to process synchronized video frames from multiple perspectives. Each frame is encoded using a pretrained vision-language encoder, and the resulting embeddings are fused to generate …

abstract applications arxiv autonomous autonomous vehicle classification control cs.ai cs.cv cs.lg driver language language models novel paper representation road safety safety semantic systems transitions type vision vision-language vision-language models