Training learned optimizers: VeLO paper EXPLAINED

Why tune optimizers hyperparameters (Adam) by hand, when one can train a neural network to behave like an optimizer and dynamically find the best update for your neural network’s weights?
In this video, we explain the work on VeLO to train an optimizer from data from previous training runs.
► Sponsor: Cohere 👉 https://t1p.de/22srn

Check out our daily #MachineLearning Quiz Questions: https://www.youtube.com/c/AICoffeeBreak/community

📜 VeELO paper: Metz, Luke, James Harrison, C. Daniel Freeman, Amil Merchant, Lucas Beyer, James Bradbury, Naman Agrawal …

adam cohere data explained information network neural network paper support training training data video work