all AI news
[R] Scaling tasks during pre-training may be > Scaling Model Parameters
May 13, 2022, 10:25 p.m. | /u/Competitive-Rub-1958
Machine Learning www.reddit.com
> ...This leads to a crucial discovery that task scaling can be an efficient alternative to model scaling; i.e., the model size has little impact on performance with an extremely large number of tasks. Our results show that task scaling can substantially improve training efficiency by 30 times in FLOPs..
*tl;dr* scale the amount of tasks as well as data, compute, hyperparameters, FLOPs and *prayers* for effectively training LLMs.
More from www.reddit.com / Machine Learning
Jobs in AI, ML, Big Data
Senior ML Researcher - 3D Geometry Processing | 3D Shape Generation | 3D Mesh Data
@ Promaton | Europe
Software Engineer, Data Platforms
@ Whatnot | San Francisco, CA, Los Angeles, CA, New York City, Phoenix, AZ, Seattle, WA, Denver, CO
Staff Data Engineer, Data Platform
@ Lilt | Indianapolis
Business Data Analyst - New Division
@ Breakthru Beverage Group | Toronto, ON, Canada
Data Operations Associate
@ iCapital | New York City, United States
Senior Data Scientist, R&D
@ Plusgrade | Toronto, Ontario