all AI news
BigScience BLOOM | 3D Parallelism Explained | Large Language Models | ML Coding Series
Sept. 20, 2022, 8:15 a.m. | Aleksa Gordić - The AI Epiphany
Aleksa Gordić - The AI Epiphany www.youtube.com
https://www.patreon.com/theaiepiphany
👨👩👧👦 Join our Discord community 👨👩👧👦
https://discord.gg/peBrCpheKE
In this 4th video of the Large Language Model series I walk you through the BigScience's BLOOM model codebase!
The main focus is on understanding the 3D parallelism:
* Pipeline parallelism
* Model parallelism
* Data parallelism
A set of beautiful engineering ideas that are behind all of the recent scaling efforts and ML success stories!
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
✅ BLOOM code: https://github.com/bigscience-workshop/Megatron-DeepSpeed
✅ Ultimate Guide to …
bigscience bloom coding explained language language models large language models series
More from www.youtube.com / Aleksa Gordić - The AI Epiphany
Ishan Misra (Meta) - Emu Video Generation
1 month, 1 week ago |
www.youtube.com
InstructPix2Pix (w/ OpenAI's Tim Brooks)
1 month, 2 weeks ago |
www.youtube.com
GPT-Fast - blazingly fast inference with PyTorch (w/ Horace He)
1 month, 3 weeks ago |
www.youtube.com
Thomas Wolf (HuggingFace) - the case for open-source!
2 months, 1 week ago |
www.youtube.com
Jeremy Howard - answer.ai, what is wrong with the academia and industry
2 months, 2 weeks ago |
www.youtube.com
LLaMA 2 w/ Thomas Scialom (LLaMA 2 lead)
4 months, 2 weeks ago |
www.youtube.com
EleutherAI Pythia w/ Hailey Schoelkopf
4 months, 3 weeks ago |
www.youtube.com
Jobs in AI, ML, Big Data
Founding AI Engineer, Agents
@ Occam AI | New York
AI Engineer Intern, Agents
@ Occam AI | US
AI Research Scientist
@ Vara | Berlin, Germany and Remote
Data Architect
@ University of Texas at Austin | Austin, TX
Data ETL Engineer
@ University of Texas at Austin | Austin, TX
Lead GNSS Data Scientist
@ Lurra Systems | Melbourne