On the Byzantine-Resilience of Distillation-Based Federated Learning

arXiv:2402.12265v1 Announce Type: new
Abstract: Federated Learning (FL) algorithms using Knowledge Distillation (KD) have received increasing attention due to their favorable properties with respect to privacy, non-i.i.d. data and communication cost. These methods depart from transmitting model parameters and, instead, communicate information about a learning task by sharing predictions on a public dataset. In this work, we study the performance of such approaches in the byzantine setting, where a subset of the clients act in an adversarial manner aiming to …

abstract algorithms arxiv attention communication cost cs.ai cs.dc cs.lg data dataset distillation federated learning information knowledge parameters predictions privacy public resilience type