Personalized Collaborative Fine-Tuning for On-Device Large Language Models

arXiv:2404.09753v1 Announce Type: cross
Abstract: We explore on-device self-supervised collaborative fine-tuning of large language models with limited local data availability. Taking inspiration from the collaborative learning community, we introduce three distinct trust-weighted gradient aggregation schemes: weight similarity-based, prediction similarity-based and validation performance-based. To minimize communication overhead, we integrate Low-Rank Adaptation (LoRA) and only exchange LoRA weight updates. Our protocols, driven by prediction and performance metrics, surpass both FedAvg and local fine-tuning methods, which is particularly evident in realistic scenarios with …

abstract aggregation arxiv availability collaborative communication community cs.cl cs.lg data explore fine-tuning gradient inspiration language language models large language large language models lora low low-rank adaptation performance personalized prediction trust type validation