Building a Performance Model for Deep Learning Recommendation Model Training on GPUs. (arXiv:2201.07821v2 [cs.LG] UPDATED)

We devise a performance model for GPU training of Deep Learning
Recommendation Models (DLRM), whose GPU utilization is low compared to other
well-optimized CV and NLP models. We show that both the device active time (the
sum of kernel runtimes) but also the device idle time are important components
of the overall device time. We therefore tackle them separately by (1) flexibly
adopting heuristic-based and ML-based kernel performance models for operators
that dominate the device active time, and (2) categorizing …

arxiv building deep learning gpus performance recommendation recommendation model training