Long Is More for Alignment: A Simple but Tough-to-Beat Baseline for Instruction Fine-Tuning

There is a consensus that instruction fine-tuning of LLMs requires high-quality data, but what are they? LIMA (NeurIPS 2023) and AlpaGasus (ICLR 2024) are state-of-the-art methods for selecting such high-quality examples, either via manual curation or using GPT-3.5-Turbo as a quality scorer. We show that the extremely simple baseline of selecting the 1,000 instructions with longest responses from standard datasets can consistently outperform these sophisticated methods according to GPT-4 and PaLM-2 as judges, while remaining competitive on the OpenLLM benchmarks …

alignment art consensus cs.cl curation data examples fine-tuning gpt gpt-3 gpt-3.5 gpt-3.5-turbo iclr llms neurips quality quality data show simple state turbo via