Analysis of Distributed Optimization Algorithms on a Real Processing-In-Memory System

arXiv:2404.07164v1 Announce Type: cross
Abstract: Machine Learning (ML) training on large-scale datasets is a very expensive and time-consuming workload. Processor-centric architectures (e.g., CPU, GPU) commonly used for modern ML training workloads are limited by the data movement bottleneck, i.e., due to repeatedly accessing the training dataset. As a result, processor-centric systems suffer from performance degradation and high energy consumption. Processing-In-Memory (PIM) is a promising solution to alleviate the data movement bottleneck by placing the computation mechanisms inside or near memory. …

abstract algorithms analysis architectures arxiv cpu cs.ai cs.ar cs.dc cs.lg data data movement dataset datasets distributed gpu in-memory machine machine learning memory modern optimization processing processor scale training type workloads