VisionGPT: Vision-Language Understanding Agent Using Generalized Multimodal Framework

arXiv:2403.09027v1 Announce Type: new
Abstract: With the emergence of large language models (LLMs) and vision foundation models, how to combine the intelligence and capacity of these open-sourced or API-available models to achieve open-world visual perception remains an open question. In this paper, we introduce VisionGPT to consolidate and automate the integration of state-of-the-art foundation models, thereby facilitating vision-language understanding and the development of vision-oriented AI. VisionGPT builds upon a generalized multimodal framework that distinguishes itself through three key features: (1) …

abstract agent api arxiv automate capacity cs.cv emergence foundation framework generalized integration intelligence language language models language understanding large language large language models llms multimodal open-world paper perception question type understanding vision visual world