Web: http://arxiv.org/abs/2110.14883

Sept. 21, 2022, 1:13 a.m. | Shenggui Li, Jiarui Fang, Zhengda Bian, Hongxin Liu, Yuliang Liu, Haichen Huang, Boxiang Wang, Yang You

cs.CV updates on arXiv.org arxiv.org

The success of Transformer models has pushed the deep learning model scale to
billions of parameters. Due to the limited memory resource of a single GPU,
However, the best practice for choosing the optimal parallel strategy is still
lacking, since it requires domain expertise in both deep learning and parallel
computing.


The Colossal-AI system addressed the above challenge by introducing a unified
interface to scale your sequential code of model training to distributed
environments. It supports parallel training methods such …

arxiv deep learning scale training

More from arxiv.org / cs.CV updates on arXiv.org

Research Scientists

@ ODU Research Foundation | Norfolk, Virginia

Embedded Systems Engineer (Robotics)

@ Neo Cybernetica | Bedford, New Hampshire

2023 Luis J. Alvarez and Admiral Grace M. Hopper Postdoc Fellowship in Computing Sciences

@ Lawrence Berkeley National Lab | San Francisco, CA

Senior Manager Data Scientist

@ NAV | Remote, US

Senior AI Research Scientist

@ Earth Species Project | Remote anywhere

Research Fellow- Center for Security and Emerging Technology (Multiple Opportunities)

@ University of California Davis | Washington, DC

Staff Fellow - Data Scientist

@ U.S. FDA/Center for Devices and Radiological Health | Silver Spring, Maryland

Staff Fellow - Senior Data Engineer

@ U.S. FDA/Center for Devices and Radiological Health | Silver Spring, Maryland

Machine Learning Engineers, Confluence Cloud

@ Atlassian | Mountain View, United States

Staff Data Engineer

@ Clio | Remote-US

Data Scientist (Analytics) - Singapore

@ Momos | Singapore, Central, Singapore

Machine Learning Scientist, Drug Discovery

@ Flagship Pioneering, Inc. | Cambridge, MA