M$^{2}$Chat: Empowering VLM for Multimodal LLM Interleaved Text-Image Generation

arXiv:2311.17963v2 Announce Type: replace
Abstract: While current LLM chatbots like GPT-4V bridge the gap between human instructions and visual representations to enable text-image generations, they still lack efficient alignment methods for high-fidelity performance on multiple downstream tasks. In this paper, we propose \textbf{$M^{2}Chat$}, a novel unified multimodal LLM framework for generating interleaved text-image conversation across various scenarios. Specifically, we propose an $M^{3}Adapter$ that efficiently integrates granular low-level visual information and high-level semantic features from multi-modality prompts. Upon the well-aligned fused …

arxiv chat cs.cv image image generation llm multimodal text text-image type vlm