Enabling Efficient Trillion Parameter Scale Training for Deep Learning Models // Tunji Ruwase

DeepSpeed: Enabling Efficient Trillion Parameter Scale Training for Deep Learning Models // Tunji Ruwase // AI in Production Conference Full Talk

// Abstract
Deep Learning (DL) is driving unprecedented progress in a wide range of Artificial Intelligence domains, including natural language processing, vision, speech, and multimodal. However, sustaining this AI revolution requires practical solutions to the extreme demands of model scaling on the compute, memory, communication and storage components of modern computing hardware. To address this challenge, we created a …

abstract artificial artificial intelligence conference deep learning deepspeed domains driving enabling however intelligence language language processing multimodal natural natural language natural language processing processing production progress scale speech talk training vision