LLMGA: Multimodal Large Language Model based Generation Assistant

arXiv:2311.16500v3 Announce Type: replace
Abstract: In this paper, we introduce a Multimodal Large Language Model-based Generation Assistant (LLMGA), leveraging the vast reservoir of knowledge and proficiency in reasoning, comprehension, and response inherent in Large Language Models (LLMs) to assist users in image generation and editing. Diverging from existing approaches where Multimodal Large Language Models (MLLMs) generate fixed-size embeddings to control Stable Diffusion (SD), our LLMGA provides a detailed language generation prompt for precise control over SD. This not only augments …

abstract arxiv assistant cs.cv editing image image generation knowledge language language model language models large language large language model large language models llms multimodal multimodal large language model paper reasoning type vast