An Experimental Design Framework for Label-Efficient Supervised Finetuning of Large Language Models

arXiv:2401.06692v2 Announce Type: replace-cross
Abstract: Supervised finetuning (SFT) on instruction datasets has played a crucial role in achieving the remarkable zero-shot generalization capabilities observed in modern large language models (LLMs). However, the annotation efforts required to produce high quality responses for instructions are becoming prohibitively expensive, especially as the number of tasks spanned by instruction datasets continues to increase. Active learning is effective in identifying useful subsets of samples to annotate from an unlabeled pool, but its high computational cost …

abstract annotation arxiv capabilities cs.ai cs.cl cs.lg datasets design experimental finetuning framework however language language models large language large language models llms modern quality responses role sft type zero-shot