DeepSpeed ZeRO++: A leap in speed for LLM and chat model training with 4X less communication

Large AI models are transforming the digital world. Generative language models like Turing-NLG, ChatGPT, and GPT-4, powered by large language models (LLMs), are incredibly versatile, capable of performing tasks like summarization, coding, and translation. Similarly, large multimodal generative models like DALL·E, Microsoft Designer, and Bing Image Creator can generate art, architecture, videos, and other digital […]

The post DeepSpeed ZeRO++: A leap in speed for LLM and chat model training with 4X less communication appeared first on Microsoft Research.

ai models chat chatgpt coding communication dall deepspeed designer digital generative generative models gpt gpt-4 language language models large language large language models llm llms microsoft multimodal nlg research blog speed summarization training translation turing world