CoDream: Exchanging dreams instead of models for federated aggregation with heterogeneous models

arXiv:2402.15968v1 Announce Type: new
Abstract: Federated Learning (FL) enables collaborative optimization of machine learning models across decentralized data by aggregating model parameters. Our approach extends this concept by aggregating "knowledge" derived from models, instead of model parameters. We present a novel framework called \codream, where clients collaboratively optimize randomly initialized data using federated optimization in the input data space, similar to how randomly initialized model parameters are optimized in FL. Our key insight is that jointly optimizing this data can …

aggregation arxiv cs.ai cs.lg dreams type