Memory-Based Optimization Methods for Model-Agnostic Meta-Learning and Personalized Federated Learning

In recent years, model-agnostic meta-learning (MAML) has become a popular research area. However, the stochastic optimization of MAML is still underdeveloped. Existing MAML algorithms rely on the “episode” idea by sampling a few tasks and data points to update the meta-model at each iteration. Nonetheless, these algorithms either fail to guarantee convergence with a constant mini-batch size or require processing a large number of tasks at every iteration, which is unsuitable for continual learning or cross-device federated learning where only …

algorithms become data federated learning iteration memory meta meta-learning model-agnostic optimization personalized popular research sampling stochastic