MLTCP: Congestion Control for DNN Training

arXiv:2402.09589v1 Announce Type: cross
Abstract: We present MLTCP, a technique to augment today's congestion control algorithms to accelerate DNN training jobs in shared GPU clusters. MLTCP enables the communication phases of jobs that compete for network bandwidth to interleave with each other, thereby utilizing the network efficiently. At the heart of MLTCP lies a very simple principle based on a key conceptual insight: DNN training flows should scale their congestion window size based on the number of bytes sent at …

abstract algorithms arxiv bandwidth communication congestion control cs.dc cs.lg cs.ni dnn gpu jobs lies network training type