Enhancing Data Quality in Federated Fine-Tuning of Foundation Models

arXiv:2403.04529v1 Announce Type: new
Abstract: In the current landscape of foundation model training, there is a significant reliance on public domain data, which is nearing exhaustion according to recent research. To further scale up, it is crucial to incorporate collaboration among multiple specialized and high-quality private domain data sources. However, the challenge of training models locally without sharing private data presents numerous obstacles in data quality control. To tackle this issue, we propose a data quality control pipeline for federated …

abstract arxiv collaboration cs.ai cs.dc cs.lg current data data quality data sources domain fine-tuning foundation foundation model however landscape multiple public public domain quality reliance research scale training type