From Complex to Simple: Enhancing Multi-Constraint Complex Instruction Following Ability of Large Language Models

arXiv:2404.15846v1 Announce Type: new
Abstract: It is imperative for Large language models (LLMs) to follow instructions with elaborate requirements (i.e. Complex Instructions Following). Yet, it remains under-explored how to enhance the ability of LLMs to follow complex instructions with multiple constraints. To bridge the gap, we initially study what training data is effective in enhancing complex constraints following abilities. We found that training LLMs with instructions containing multiple constraints enhances their understanding of complex instructions, especially those with lower complexity …

abstract arxiv bridge constraints cs.cl gap language language models large language large language models llms multiple requirements simple type