NExT-GPT: Any-to-Any Multimodal LLM

arXiv:2309.05519v3 Announce Type: replace-cross
Abstract: While recently Multimodal Large Language Models (MM-LLMs) have made exciting strides, they mostly fall prey to the limitation of only input-side multimodal understanding, without the ability to produce content in multiple modalities. As we humans always perceive the world and communicate with people through various modalities, developing any-to-any MM-LLMs capable of accepting and delivering content in any modality becomes essential to human-level AI. To fill the gap, we present an end-to-end general-purpose any-to-any MM-LLM system, …

arxiv cs.ai cs.cl cs.lg gpt llm multimodal next replace type