Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Survey

arXiv:2403.14608v1 Announce Type: new
Abstract: Large models represent a groundbreaking advancement in multiple application fields, enabling remarkable achievements across various tasks. However, their unprecedented scale comes with significant computational costs. These models, often consisting of billions of parameters, require vast amounts of computational resources for execution. Especially, the expansive scale and computational demands pose considerable challenges when customizing them for particular downstream tasks, particularly over the hardware platforms constrained by computational capabilities. Parameter Efficient Fine-Tuning (PEFT) provides a practical solution …

abstract advancement application arxiv computational costs cs.lg enabling fields fine-tuning groundbreaking however large models multiple parameters resources scale survey tasks type vast