Automated Data Curation for Robust Language Model Fine-Tuning

arXiv:2403.12776v1 Announce Type: new
Abstract: Large Language Models have become the de facto approach to sequence-to-sequence text generation tasks, but for specialized tasks/domains, a pretrained LLM lacks specific capabilities to produce accurate or well-formatted responses. Supervised fine-tuning specializes a LLM by training it on dataset of example prompts with target responses, but real-world data tends to be noisy. While many fine-tuning algorithms exist, here we consider a \emph{data-centric AI} perspective on LLM fine-tuning, studying how to \emph{systematically} curate the training …

abstract arxiv automated become capabilities cs.cl curation data data curation dataset domains example fine-tuning language language model language models large language large language models llm model fine-tuning prompts responses robust supervised fine-tuning tasks text text generation training type