Automated Federated Pipeline for Parameter-Efficient Fine-Tuning of Large Language Models

arXiv:2404.06448v1 Announce Type: new
Abstract: Recently, there has been a surge in the development of advanced intelligent generative content (AIGC), especially large language models (LLMs). However, for many downstream tasks, it is necessary to fine-tune LLMs using private data. While federated learning offers a promising privacy-preserving solution to LLM fine-tuning, the substantial size of an LLM, combined with high computational and communication demands, makes it hard to apply to downstream tasks. More importantly, private edge servers often possess varying computing …

abstract advanced aigc arxiv automated cs.ai cs.lg data development federated learning fine-tuning generative however intelligent language language models large language large language models llm llms pipeline privacy private data solution tasks type