Large Model Training and Inference with DeepSpeed // Samyam Rajbhandari // LLMs in Prod Conference

//Abstract
In the last few years, DeepSpeed has released numerous technologies for training and inference of large models, transforming the large model training landscape from a system perspective. Technologies like ZeRO, and 3D-Parallelism have become the building blocks for training large models at scale, powering LLMs like Bloom-176B, Megatron-Turing 530B, and many others. Heterogenous memory training systems like ZeRO-Offload and ZeRO-Infinity have democratized LLMs by making them accessible with limited resources. DeepSpeed-Inference and DeepSpeed-MII have made it easy to apply …

abstract become building conference deepspeed inference landscape large models llms perspective prod scale technologies training