all AI news
[R] Mixtures of Experts Unlock Parameter Scaling for Deep RL
Feb. 16, 2024, 3:02 a.m. | /u/OwnAd9305
Machine Learning www.reddit.com
> The recent rapid progress in (self) supervised learning models is in large part predicted by empirical scaling laws: a model’s performance scales proportionally to its size. Analogous scaling laws remain elusive for reinforcement learning domains, however, where increasing the parameter count of a model often hurts its final performance. In this paper, we demonstrate that incorporating Mixture-of-Expert (MoE) modules, and in particular Soft MoEs (Puigcerver et al., 2023), into value-based networks results in more parameter-scalable models, evidenced by …
abstract count deep rl domains experts laws machinelearning part performance progress reinforcement reinforcement learning scaling s performance supervised learning
More from www.reddit.com / Machine Learning
Jobs in AI, ML, Big Data
ML/AI Engineer / NLP Expert - Custom LLM Development (x/f/m)
@ HelloBetter | Remote
Doctoral Researcher (m/f/div) in Automated Processing of Bioimages
@ Leibniz Institute for Natural Product Research and Infection Biology (Leibniz-HKI) | Jena
Seeking Developers and Engineers for AI T-Shirt Generator Project
@ Chevon Hicks | Remote
Cloud Data Platform Engineer
@ First Central | Home Office (Remote)
Associate Director, Data Science
@ MSD | USA - New Jersey - Rahway
Data Scientist Sr.
@ MSD | CHL - Santiago - Santiago (Calle Mariano)