Personalized Federated Instruction Tuning via Neural Architecture Search

arXiv:2402.16919v1 Announce Type: new
Abstract: Federated Instruction Tuning (FIT) has shown the ability to achieve collaborative model instruction tuning among massive data owners without sharing private data. However, it still faces two key challenges, i.e., data and resource heterogeneity. Due to the varying data distribution and preferences among data owners, FIT cannot adapt to the personalized data of individual owners. Moreover, clients with superior computational abilities are constrained since they need to maintain the same fine-tuning architecture as the weaker …

abstract architecture arxiv challenges collaborative cs.lg data distribution key massive neural architecture search personalized private data search type via