Self-Distillation Bridges Distribution Gap in Language Model Fine-Tuning

arXiv:2402.13669v1 Announce Type: new
Abstract: The surge in Large Language Models (LLMs) has revolutionized natural language processing, but fine-tuning them for specific tasks often encounters challenges in balancing performance and preserving general instruction-following abilities. In this paper, we posit that the distribution gap between task datasets and the LLMs serves as the primary underlying cause. To address the problem, we introduce Self-Distillation Fine-Tuning (SDFT), a novel approach that bridges the distribution gap by guiding fine-tuning with a distilled dataset generated …

arxiv cs.cl distillation distribution fine-tuning gap language language model model fine-tuning type