Re-weighted gradient descent via distributionally robust optimization

Deep neural networks (DNNs) have become essential for solving a wide range of tasks, from standard supervised learning (image classification using ViT) to meta-learning. The most commonly-used paradigm for learning DNNs is empirical risk minimization (ERM), which aims to identify a network that minimizes the average loss on training data points. Several algorithms, including stochastic gradient descent (SGD), Adam, and Adagrad, have …

become classification deep learning erm google google research gradient identify image kumar machine learning meta meta-learning networks neural networks optimization paradigm research researcher research scientist risk standard supervised learning tasks vit