Otter: A Multi-Modal Model with In-Context Instruction Tuning. (arXiv:2305.03726v1 [cs.CV])

Large language models (LLMs) have demonstrated significant universal
capabilities as few/zero-shot learners in various tasks due to their
pre-training on vast amounts of text data, as exemplified by GPT-3, which
boosted to InstrctGPT and ChatGPT, effectively following natural language
instructions to accomplish real-world tasks. In this paper, we propose to
introduce instruction tuning into multi-modal models, motivated by the Flamingo
model's upstream interleaved format pretraining dataset. We adopt a similar
approach to construct our MultI-Modal In-Context Instruction Tuning (MIMIC-IT)
dataset. …

arxiv chatgpt context data gpt gpt-3 language language models large language models llms natural natural language otter paper pre-training text training world