TernaryVote: Differentially Private, Communication Efficient, and Byzantine Resilient Distributed Optimization on Heterogeneous Data

arXiv:2402.10816v1 Announce Type: new
Abstract: Distributed training of deep neural networks faces three critical challenges: privacy preservation, communication efficiency, and robustness to fault and adversarial behaviors. Although significant research efforts have been devoted to addressing these challenges independently, their synthesis remains less explored. In this paper, we propose TernaryVote, which combines a ternary compressor and the majority vote mechanism to realize differential privacy, gradient compression, and Byzantine resilience simultaneously. We theoretically quantify the privacy guarantee through the lens of the …

abstract adversarial arxiv challenges communication cs.cr cs.dc cs.lg data distributed eess.sp efficiency networks neural networks optimization paper preservation privacy research resilient robustness synthesis training type