How Can We Train Deep Learning Models Across Clouds and Continents? An Experimental Study

This paper aims to answer the question: Can deep learning models be cost-efficiently trained on a global market of spot VMs spanning different data centers and cloud providers? To provide guidance, we extensively evaluate the cost and throughput implications of training in different zones, continents, and clouds for representative CV, NLP, and ASR models. To expand the current training options further, we compare the scalability potential for hybrid-cloud scenarios by adding cloud resources to on-premise hardware to improve training throughput. …

cloud cloud providers cost cs.dc cs.lg cs.ni cs.pf data data centers deep learning experimental global guidance paper question spot study train training