Instruction Fine-Tuning: Does Prompt Loss Matter?

arXiv:2401.13586v2 Announce Type: replace
Abstract: We present a study analyzing the effects of prompt loss weighting (PLW) on supervised instruction fine-tuning. We recreated Stanford's Alpaca experiment with both LLaMA 1 and LLaMA 2 and multiple instruction datasets. We found that performance of models fine-tuned on our short-completion dataset had a statistically significant negative quadratic relationship with PLW, but performance of models fine-tuned on medium- and long-completion data did not show any relationship with PLW. I.e., prompt loss can be safely …

abstract alpaca arxiv cs.ai cs.cl cs.lg dataset datasets effects experiment fine-tuning found llama llama 2 loss matter multiple negative performance prompt stanford study type