Web: https://www.youtube.com/watch?v=cdNLtMszdLs

May 12, 2022, 4:33 p.m. | TensorFlow

TensorFlow youtube.com

Discover several different distribution strategies and related concepts for data and model parallel training. Walk through an example of training a 39 billion parameter language model on TPUs, and conclude with the challenges and best practices of orchestrating large scale language model training.

Resource:
TensorFlow website → https://goo.gle/3KejoUZ

Speakers: Nikita Namjoshi, Vaibhav Singh

Watch more:
All Google I/O 2022 Sessions → https://goo.gle/IO22_AllSessions
ML/AI at I/O 2022 playlist → https://goo.gle/IO22_ML-AI
All Google I/O 2022 technical sessions → https://goo.gle/IO22_Sessions

Subscribe to TensorFlow …

distributed model tips training tricks

Director, Applied Mathematics & Computational Research Division

@ Lawrence Berkeley National Lab | Berkeley, Ca

Business Data Analyst

@ MainStreet Family Care | Birmingham, AL

Assistant/Associate Professor of the Practice in Business Analytics

@ Georgetown University McDonough School of Business | Washington DC

Senior Data Science Writer

@ NannyML | Remote

Director of AI/ML Engineering

@ Armis Industries | Remote (US only), St. Louis, California

Digital Analytics Manager

@ Patagonia | Ventura, California