Preserving In-Context Learning ability in Large Language Model Fine-tuning. (arXiv:2211.00635v1 [cs.CL])

Pretrained large language models (LLMs) are strong in-context learners that
are able to perform few-shot learning without changing model parameters.
However, as we show, fine-tuning an LLM on any specific task generally destroys
its in-context ability. We discover an important cause of this loss, format
specialization, where the model overfits to the format of the fine-tuned task
and is unable to output anything beyond this format. We further show that
format specialization happens at the beginning of fine-tuning. To solve …

arxiv context fine-tuning language language model large language model model fine-tuning