DPPA: Pruning Method for Large Language Model to Model Merging

arXiv:2403.02799v1 Announce Type: new
Abstract: Model merging is to combine fine-tuned models derived from multiple domains, with the intent of enhancing the model's proficiency across various domains. The principal concern is the resolution of parameter conflicts. A substantial amount of existing research remedy this issue during the merging stage, with the latest study focusing on resolving this issue throughout the pruning stage. The DARE approach has exhibited promising outcomes when applied to a simplistic fine-tuned model. However, the efficacy of …

abstract arxiv cs.ai cs.cl domains issue language language model large language large language model merging multiple pruning research stage type