Towards a Flexible and High-Fidelity Approach to Distributed DNN Training Emulation

arXiv:2405.02969v1 Announce Type: new
Abstract: We propose NeuronaBox, a flexible, user-friendly, and high-fidelity approach to emulate DNN training workloads. We argue that to accurately observe performance, it is possible to execute the training workload on a subset of real nodes and emulate the networked execution environment along with the collective communication operations. Initial results from a proof-of-concept implementation show that NeuronaBox replicates the behavior of actual systems with high accuracy, with an error margin of less than 1% between the …

abstract arxiv cs.dc cs.lg distributed dnn environment fidelity nodes observe performance training type workloads