all AI news
[P] Farewell, CUDA OOM: Automatic Gradient Accumulation
June 23, 2022, 9:28 p.m. | /u/ffast-math
Machine Learning www.reddit.com
If you've trained a lot of neural nets, you probably know the pain of getting CUDA OOM errors and iteratively tuning your batch size to avoid them.
Which is why I'm excited to announce that we (MosaicML) just released an automatic way to avoid these errors. Namely, we just added [automatic gradient accumulation](https://docs.mosaicml.com/en/latest/notes/auto_grad_accum.html) to [Composer](https://github.com/mosaicml/composer), our open source library for faster + easier neural net training.
If you're not familiar with gradient accumulation, it's like tuning the batch …
More from www.reddit.com / Machine Learning
Jobs in AI, ML, Big Data
Senior ML Researcher - 3D Geometry Processing | 3D Shape Generation | 3D Mesh Data
@ Promaton | Europe
Data Architect
@ Western Digital | San Jose, CA, United States
Senior Data Scientist GenAI (m/w/d)
@ Deutsche Telekom | Bonn, Deutschland
Senior Data Engineer, Telco (Remote)
@ Lightci | Toronto, Ontario
Consultant Data Architect/Engineer H/F - Innovative Tech
@ Devoteam | Lyon, France
(Senior) ML Engineer / Software Engineer Machine Learning & AI (m/f/x) onsite or remote (in Germany or Austria)
@ Scalable GmbH | Wien, Germany