VisionGPT: LLM-Assisted Real-Time Anomaly Detection for Safe Visual Navigation

arXiv:2403.12415v1 Announce Type: new
Abstract: This paper explores the potential of Large Language Models(LLMs) in zero-shot anomaly detection for safe visual navigation. With the assistance of the state-of-the-art real-time open-world object detection model Yolo-World and specialized prompts, the proposed framework can identify anomalies within camera-captured frames that include any possible obstacles, then generate concise, audio-delivered descriptions emphasizing abnormalities, assist in safe visual navigation in complex circumstances. Moreover, our proposed framework leverages the advantages of LLMs and the open-vocabulary object detection …

abstract anomaly anomaly detection art arxiv cs.cv cs.hc detection framework identify language language models large language large language models llm llms navigation object obstacles open-world paper prompts real-time state type visual visual navigation world yolo zero-shot