Distributed Learning based on 1-Bit Gradient Coding in the Presence of Stragglers

arXiv:2403.14716v1 Announce Type: new
Abstract: This paper considers the problem of distributed learning (DL) in the presence of stragglers. For this problem, DL methods based on gradient coding have been widely investigated, which redundantly distribute the training data to the workers to guarantee convergence when some workers are stragglers. However, these methods require the workers to transmit real-valued vectors during the process of learning, which induces very high communication burden. To overcome this drawback, we propose a novel DL method …

abstract arxiv coding convergence cs.dc cs.lg data distributed distributed learning gradient paper training training data type workers