What Makes Good Data for Alignment? A Comprehensive Study of Automatic Data Selection in Instruction Tuning

arXiv:2312.15685v2 Announce Type: replace-cross
Abstract: Instruction tuning is a standard technique employed to align large language models to end tasks and user preferences after the initial pretraining phase. Recent research indicates the critical role of data engineering in instruction tuning -- when appropriately selected, only limited data is necessary to achieve superior performance. However, we still lack a principled understanding of what makes good instruction tuning data for alignment, and how we should select data automatically and effectively. In this …

abstract alignment arxiv cs.ai cs.cl cs.lg data data engineering engineering good language language models large language large language models pretraining research role standard study tasks type