HFT: Half Fine-Tuning for Large Language Models

arXiv:2404.18466v1 Announce Type: new
Abstract: Large language models (LLMs) with one or more fine-tuning phases have become a necessary step to unlock various capabilities, enabling LLMs to follow natural language instructions or align with human preferences. However, it carries the risk of catastrophic forgetting during sequential training, the parametric knowledge or the ability learned in previous stages may be overwhelmed by incoming training data. In this paper, we find that by regularly resetting partial parameters, LLMs can restore some of …

abstract arxiv become capabilities catastrophic forgetting cs.cl enabling fine-tuning however human knowledge language language models large language large language models llms natural natural language parametric risk training type