Fine-tuning Large Language Models with Sequential Instructions

arXiv:2403.07794v1 Announce Type: new
Abstract: Large language models (LLMs) struggle to follow a sequence of instructions in a single query as they may ignore or misinterpret part of it. This impairs their performance in complex problems whose solution requires multiple intermediate steps, such as multilingual (translate then answer) and multimodal (caption then answer) tasks. We empirically verify this with open-source LLMs as large as LLaMA-2 70B and Mixtral-8x7B. Targeting the scarcity of sequential instructions in present-day data, we propose sequential …

abstract arxiv cs.cl fine-tuning intermediate language language models large language large language models llms multilingual multimodal multiple part performance query solution struggle translate type