Communication-Efficient Adam-Type Algorithms for Distributed Data Mining. (arXiv:2210.07454v1 [cs.LG])

Distributed data mining is an emerging research topic to effectively and
efficiently address hard data mining tasks using big data, which are
partitioned and computed on different worker nodes, instead of one centralized
server. Nevertheless, distributed learning methods often suffer from the
communication bottleneck when the network bandwidth is limited or the size of
model is large. To solve this critical issue, many gradient compression methods
have been proposed recently to reduce the communication cost for multiple
optimization algorithms. However, …

adam algorithms arxiv communication data data mining distributed distributed data mining type